-- Logs begin at Mon 2021-05-24 10:38:28 UTC, end at Mon 2021-05-24 11:39:47 UTC. -- May 24 11:07:37.352612 dv-cinder-784092-9-iscsi systemd[1]: Started Devstack devstack@q-svc.service. May 24 11:07:38.311992 dv-cinder-784092-9-iscsi neutron-server[91712]: Beware of the coming changes in the project packaging May 24 11:07:38.312724 dv-cinder-784092-9-iscsi neutron-server[91712]: https://github.com/svinota/pyroute2/discussions/786 May 24 11:07:39.762333 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.common.config [-] Logging enabled! May 24 11:07:39.763015 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.common.config [-] /usr/local/bin/neutron-server version 18.1.0.dev231 May 24 11:07:39.763108 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=91712) setup_logging /opt/stack/neutron/neutron/common/config.py:112}} May 24 11:07:39.766430 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.common.config [-] Logging enabled! May 24 11:07:39.766685 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.common.config [-] /usr/local/bin/neutron-server version 18.1.0.dev231 May 24 11:07:39.766995 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=91712) setup_logging /opt/stack/neutron/neutron/common/config.py:112}} May 24 11:07:39.767931 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ******************************************************************************** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 11:07:39.768165 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] Configuration options gathered from: {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 11:07:39.768519 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 11:07:39.768763 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 11:07:39.768990 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ================================================================================ {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 11:07:39.769334 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] allow_bulk = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.769713 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] allow_overlapping_ips = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.770015 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] api_extensions_path = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.770244 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] api_paste_config = api-paste.ini {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.770539 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] api_workers = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.770832 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] auth_strategy = keystone {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.771117 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] backlog = 4096 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.771410 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] base_mac = fa:16:3e:00:00:00 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.771711 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] bind_host = 0.0.0.0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.771996 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] bind_port = 9696 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.772298 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] client_socket_timeout = 900 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.772566 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] config_dir = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.772801 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.773083 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] config_source = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.773481 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] control_exchange = neutron {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.773775 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] core_plugin = ml2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.774000 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] debug = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.774284 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] default_availability_zones = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.774715 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.775143 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] dhcp_agent_notification = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.775432 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] dhcp_lease_duration = 86400 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.775726 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] dns_domain = openstacklocal {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.776007 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] enable_traditional_dhcp = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.776306 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] external_dns_driver = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.776590 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] filter_validation = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.776878 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] global_physnet_mtu = 1500 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.777236 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] host = dv-cinder-784092-9-iscsi {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.777517 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] http_retries = 3 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.777808 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] instance_format = [instance: %(uuid)s] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.778097 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.778385 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ipam_driver = internal {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.778662 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ipv6_pd_enabled = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.778942 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] log_config_append = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.779230 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.779507 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] log_dir = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.779783 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] log_file = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.780060 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] log_rotate_interval = 1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.780350 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] log_rotate_interval_type = days {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.780639 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] log_rotation_type = none {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.780966 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.781319 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.781619 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.781845 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.782138 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.782418 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] max_dns_nameservers = 5 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.782699 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] max_header_line = 16384 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.782979 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] max_logfile_count = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.783257 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] max_logfile_size_mb = 200 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.783534 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] max_subnet_host_routes = 20 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.783807 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] network_link_prefix = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.784093 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] notify_nova_on_port_data_changes = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.784378 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] notify_nova_on_port_status_changes = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.784666 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] pagination_max_limit = -1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.784943 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] periodic_fuzzy_delay = 5 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.785251 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] periodic_interval = 40 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.785529 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] publish_errors = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.785803 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] rate_limit_burst = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.786092 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] rate_limit_except_level = CRITICAL {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.786368 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] rate_limit_interval = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.786642 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] retry_until_window = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.786914 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] rpc_response_max_timeout = 600 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.787202 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] rpc_state_report_workers = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.787473 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] rpc_workers = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.787747 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] send_events_interval = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.788047 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.788336 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] setproctitle = on {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.788619 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] state_path = /opt/stack/data/neutron {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.788908 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] syslog_log_facility = LOG_USER {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.789209 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] tcp_keepidle = 600 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.789497 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] transport_url = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.789770 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] use_eventlog = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.790046 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] use_journal = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.790318 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] use_json = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.790589 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] use_ssl = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.790862 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] use_stderr = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.791146 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] use_syslog = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.791421 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] vlan_transparent = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.791699 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] watch_log_file = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.792488 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] wsgi_default_pool_size = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.792760 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] wsgi_keep_alive = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.793049 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:39.793372 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_concurrency.disable_process_locking = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.793671 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.793977 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] profiler.connection_string = messaging:// {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.794262 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] profiler.enabled = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.794560 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] profiler.es_doc_type = notification {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.794840 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] profiler.es_scroll_size = 10000 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.795132 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] profiler.es_scroll_time = 2m {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.795415 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] profiler.filter_error_trace = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.795707 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] profiler.hmac_keys = SECRET_KEY {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.796003 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] profiler.sentinel_service_name = mymaster {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.796295 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] profiler.socket_timeout = 0.1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.796575 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] profiler.trace_sqlalchemy = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.796862 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_policy.enforce_new_defaults = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.797192 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_policy.enforce_scope = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.797555 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_policy.policy_default_rule = default {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.797865 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.798166 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.798461 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.798740 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.799019 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.799297 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.799579 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.799868 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] nova.auth_section = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.800162 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] nova.auth_type = password {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.800455 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.800732 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] nova.certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.801011 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] nova.collect_timing = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.801327 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] nova.endpoint_type = public {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.801610 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] nova.insecure = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.801892 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] nova.keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.802170 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] nova.live_migration_events = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.802464 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] nova.region_name = RegionOne {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.802745 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] nova.split_loggers = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.803019 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] nova.timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.803304 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] placement.auth_section = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.803586 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] placement.auth_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.803862 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] placement.cafile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.804136 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] placement.certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.804416 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] placement.collect_timing = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.804710 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] placement.endpoint_type = public {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.804988 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] placement.insecure = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.805296 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] placement.keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.805618 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] placement.region_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.805855 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] placement.split_loggers = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.806129 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] placement.timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.806414 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.auth_section = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.806691 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.auth_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.806964 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.cafile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.807236 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.807514 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.collect_timing = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.808337 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.connect_retries = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.808612 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.connect_retry_delay = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.808892 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.enable_notifications = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.809192 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.endpoint_override = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.809471 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.insecure = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.809747 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.interface = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.810021 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.810293 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.max_version = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.810566 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.min_version = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.810838 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.region_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.811109 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.service_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.811380 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.service_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.811657 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.split_loggers = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.811935 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.status_code_retries = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.812207 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.status_code_retry_delay = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.812476 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.812748 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.valid_interfaces = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.813023 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ironic.version = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.813411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] service_providers.service_provider = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.813717 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.814001 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.814283 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.814562 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.814841 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.815121 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.815397 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.815676 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.815961 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.816237 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.816529 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.816814 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.817114 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.817407 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.817686 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.817983 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.818264 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.818547 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.818824 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.819103 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.819395 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.819681 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.819976 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.820274 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.820572 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.820869 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_version = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.821191 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_notifications.driver = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.821729 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_notifications.retry = -1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.822042 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.822326 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] oslo_messaging_notifications.transport_url = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:39.822697 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.common.config [-] ******************************************************************************** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 11:07:39.822949 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo.service.wsgi [-] Loading app neutron from /etc/neutron/api-paste.ini {{(pid=91712) load_app /usr/local/lib/python3.8/dist-packages/oslo_service/wsgi.py:349}} May 24 11:07:40.169944 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=91712) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:07:40.170220 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" released by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.001s {{(pid=91712) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:07:40.170507 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=91712) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:07:40.170861 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.manager [-] Loading core plugin: ml2 May 24 11:07:40.561390 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.561685 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.561954 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.563152 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.563424 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.563681 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.597934 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.598213 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.598482 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.612154 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.612426 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.612686 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.642089 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.642363 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.642623 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.678858 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.679132 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.679393 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.910330 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.910960 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.910960 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment precommit_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.911146 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: network precommit_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:40.911823 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.035861 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.036176 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.036455 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.036727 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.036996 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.037343 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.037602 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.037856 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.038108 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.038389 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.038647 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network precommit_delete 0 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.038908 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port provisioning_complete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.039175 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.039430 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.039692 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > agent after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.039957 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.040212 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet precommit_delete 0 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.040622 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.040876 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.041186 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.041533 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.041783 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.042640 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.042921 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.043169 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.043411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.043795 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=91712) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 11:07:41.044032 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=91712) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 11:07:41.044270 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=91712) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 11:07:41.044503 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=91712) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 11:07:41.044736 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=91712) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 11:07:41.044968 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=91712) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 11:07:41.045360 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] May 24 11:07:41.048498 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['physnet1'] May 24 11:07:41.052342 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete May 24 11:07:41.056013 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict([('physnet1', [(500, 549)])]) May 24 11:07:41.057663 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'gre', 'local', 'vlan', 'vxlan'] May 24 11:07:41.058105 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'gre', 'local', 'vlan', 'vxlan']) May 24 11:07:41.058469 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [-] Tenant network_types: ['vlan'] May 24 11:07:41.058848 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security'] May 24 11:07:41.061853 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security'] May 24 11:07:41.062108 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security'] May 24 11:07:41.062458 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['openvswitch', 'hyperv'] May 24 11:07:41.090216 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.drivers.openvswitch.mech_driver.mech_openvswitch [-] Open vSwitch agent's supported_vnic_types: ['normal', 'direct', 'smart-nic', 'vdpa'] May 24 11:07:41.090666 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > qos_plugin after_init 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.090963 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.services.qos.drivers.openvswitch.driver [-] Open vSwitch QoS driver registered {{(pid=91712) register /opt/stack/neutron/neutron/services/qos/drivers/openvswitch/driver.py:82}} May 24 11:07:41.091319 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > logging-plugin after_init 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.092652 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.services.logapi.drivers.openvswitch.driver [-] Open vSwitch logging driver registered {{(pid=91712) register /opt/stack/neutron/neutron/services/logapi/drivers/openvswitch/driver.py:69}} May 24 11:07:41.092935 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['openvswitch', 'hyperv'] May 24 11:07:41.093211 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['openvswitch', 'hyperv'] May 24 11:07:41.096370 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip before_response 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.096642 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_response 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.166663 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat' May 24 11:07:41.166852 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete May 24 11:07:41.167635 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve' May 24 11:07:41.168008 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 1000)] May 24 11:07:41.197769 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_db.sqlalchemy.engines [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91712) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 11:07:41.522905 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Initializing driver for type 'gre' May 24 11:07:41.523536 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.drivers.type_tunnel [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] gre ID ranges: [(1, 1000)] May 24 11:07:41.575950 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Initializing driver for type 'local' May 24 11:07:41.576286 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Initializing driver for type 'vlan' May 24 11:07:41.589400 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] VlanTypeDriver initialization complete May 24 11:07:41.589747 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Initializing driver for type 'vxlan' May 24 11:07:41.590241 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.drivers.type_tunnel [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] vxlan ID ranges: [(1, 1000)] May 24 11:07:41.640809 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Initializing extension driver 'port_security' May 24 11:07:41.641215 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.extensions.port_security [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] PortSecurityExtensionDriver initialization complete May 24 11:07:41.641518 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Initializing mechanism driver 'openvswitch' May 24 11:07:41.641853 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Initializing mechanism driver 'hyperv' May 24 11:07:41.648216 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.plugins.ml2.plugin [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=91712) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:07:41.649001 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > port after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.649389 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > port after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.649737 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > port after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.650123 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > subnet after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.650469 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > subnet after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.650807 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > subnet after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.651186 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > network after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.652119 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > network after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.652456 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > network after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.652827 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > security_group after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.653195 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > security_group after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.653533 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > security_group after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.653906 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > security_group_rule after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.654245 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > security_group_rule after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.654574 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > security_group_rule after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.654941 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > address_group after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.655271 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > address_group after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.655600 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > address_group after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.655930 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] ML2 OVO RPC backend initialized. {{(pid=91712) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:160}} May 24 11:07:41.656619 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router_interface after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.656963 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router_interface after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.657411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > network before_response 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.657750 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > networks before_response 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.658090 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > port before_response 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.658424 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > ports before_response 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.658755 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > subnet before_response 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.659086 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > subnets before_response 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.659424 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > network after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.659752 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > network after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.660079 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > network after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.660408 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > port after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.660734 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > port after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.661057 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > port after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.661414 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > subnet after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.661738 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > subnet after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.662063 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > subnet after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.662620 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.plugin [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Modular L2 Plugin initialization complete May 24 11:07:41.662939 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading services supported by the core plugin {{(pid=91712) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:168}} May 24 11:07:41.663255 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.plugins.ml2.managers [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Got port-security extension from driver 'port_security' May 24 11:07:41.663709 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.extensions.vlantransparent [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Disabled vlantransparent extension. May 24 11:07:41.664185 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading service plugins: ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=91712) _load_service_plugins /opt/stack/neutron/neutron/manager.py:194}} May 24 11:07:41.664489 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading Plugin: router May 24 11:07:41.701702 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router_interface before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.702093 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router after_create 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.702459 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router before_create 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.702823 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router_interface before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.703183 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router_interface after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.703528 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router after_delete 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.703879 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > floatingip after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.704232 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router after_create 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.704569 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router after_update 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.704913 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router after_update 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.705277 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router_gateway after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.705630 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router after_update 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.706027 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router precommit_update 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.706389 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: port after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.706723 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: port after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.707053 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: subnet after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.707385 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: subnetpool_address_scope after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.707724 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > port precommit_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.708066 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router precommit_create 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.708401 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: port before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.708734 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router precommit_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.709106 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router after_update 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.709442 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router precommit_delete 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.709783 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router precommit_create 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.710137 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router_interface after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.710486 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router precommit_update 55549900 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.710844 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > network after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.711679 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Tracking information for resource: router configured {{(pid=91712) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 11:07:41.712002 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Tracking information for resource: floatingip configured {{(pid=91712) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 24 11:07:41.716852 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.db.l3_agentschedulers_db [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Skipping period L3 agent status check because automatic router rescheduling is disabled. May 24 11:07:41.717447 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: port after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.717803 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: port after_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.718147 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: port after_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.718527 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: port after_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.719258 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router before_create 55550100 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.719614 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router precommit_delete 55550100 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.719967 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router precommit_create 55550100 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.720322 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > router precommit_update 55550100 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.724819 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.services.provider_configuration [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service providers = [] {{(pid=91712) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:171}} May 24 11:07:41.725209 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.db.servicetype_db [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=91712) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:42}} May 24 11:07:41.728046 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.services.service_base [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded 'neutron.services.l3_router.service_providers.dvrha.DvrHaDriver' provider for service L3_ROUTER_NAT {{(pid=91712) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 24 11:07:41.728393 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.services.service_base [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded 'neutron.services.l3_router.service_providers.dvr.DvrDriver' provider for service L3_ROUTER_NAT {{(pid=91712) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 24 11:07:41.728731 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.services.service_base [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded 'neutron.services.l3_router.service_providers.ha.HaDriver' provider for service L3_ROUTER_NAT {{(pid=91712) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 24 11:07:41.729655 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.services.service_base [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded 'neutron.services.l3_router.service_providers.single_node.SingleNodeDriver' provider for service L3_ROUTER_NAT {{(pid=91712) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 24 11:07:41.730024 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.services.service_base [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Default provider is not specified for service type L3_ROUTER_NAT May 24 11:07:41.730554 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding between (L2) Neutron networks and access to external networks via a NAT gateway. {{(pid=91712) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 11:07:41.730870 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading Plugin: neutron.services.metering.metering_plugin.MeteringPlugin May 24 11:07:41.741129 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Successfully loaded METERING plugin. Description: METERING {{(pid=91712) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 11:07:41.741464 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading Plugin: auto_allocate May 24 11:07:41.747654 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: network precommit_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.748018 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: network precommit_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.748525 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=91712) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 11:07:41.748837 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading Plugin: tag May 24 11:07:41.755543 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=91712) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 11:07:41.755865 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading Plugin: timestamp May 24 11:07:41.759582 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=91712) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 11:07:41.759914 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading Plugin: network_ip_availability May 24 11:07:41.765676 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > rbac-policy before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.766041 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > rbac-policy before_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.766393 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.767088 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=91712) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 11:07:41.767402 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading Plugin: flavors May 24 11:07:41.772188 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=91712) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 11:07:41.772513 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading Plugin: revisions May 24 11:07:41.777649 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=91712) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 24 11:07:41.777977 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_concurrency.lockutils [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Lock "manager" released by "neutron.manager.NeutronManager._create_instance" :: held 1.607s {{(pid=91712) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:07:41.778696 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] get_extension_paths = OrderedDict([('/opt/stack/neutron/neutron/extensions', 1)]) {{(pid=91712) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:635}} May 24 11:07:41.779051 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Initializing extension manager. May 24 11:07:41.779604 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: __init__.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.779883 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: __pycache__ {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.780222 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: _standard_attr_segment_lib.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.780544 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: address_group.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.781314 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.781877 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: address-group May 24 11:07:41.782195 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: address_scope.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.782961 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.783422 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: address-scope May 24 11:07:41.783747 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: admin_state_down_before_update.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.784657 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.785194 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: router-admin-state-down-before-update May 24 11:07:41.785508 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: agent.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.786187 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.786688 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: agent May 24 11:07:41.787025 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: agent_resources_synced.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.787892 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.788394 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: agent-resources-synced May 24 11:07:41.788711 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: allowedaddresspairs.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.790481 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.791001 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: allowed-address-pairs May 24 11:07:41.791329 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: auto_allocated_topology.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.792992 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.793580 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: auto-allocated-topology May 24 11:07:41.793864 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: availability_zone.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.794565 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.795063 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: availability_zone May 24 11:07:41.795355 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: availability_zone_filter.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.796246 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.796747 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: availability_zone_filter May 24 11:07:41.797064 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: data_plane_status.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.797971 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.798469 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension data-plane-status not supported by any of loaded plugins May 24 11:07:41.798788 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: default_subnetpools.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.799674 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.800180 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: default-subnetpools May 24 11:07:41.800501 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: dhcpagentscheduler.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.801314 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.801816 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: dhcp_agent_scheduler May 24 11:07:41.802135 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: dns.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.803017 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.803518 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension dns-integration not supported by any of loaded plugins May 24 11:07:41.803839 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: dns_domain_ports.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.804721 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.805253 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension dns-domain-ports not supported by any of loaded plugins May 24 11:07:41.805572 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: dvr.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.806220 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.806718 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: dvr May 24 11:07:41.807031 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: empty_string_filtering.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.807956 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.808457 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: empty-string-filtering May 24 11:07:41.808770 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.809678 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.810179 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins May 24 11:07:41.810492 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.811366 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.811858 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins May 24 11:07:41.812218 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: external_net.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.813045 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.813571 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: external-net May 24 11:07:41.813887 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: extra_dhcp_opt.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.814774 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.815277 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: extra_dhcp_opt May 24 11:07:41.815594 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: extraroute.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.816476 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.816980 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: extraroute May 24 11:07:41.817328 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: extraroute_atomic.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.818330 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.818834 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: extraroute-atomic May 24 11:07:41.819150 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: filter_validation.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.819793 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.820295 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: filter-validation May 24 11:07:41.820607 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: fip_pf_description.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.821958 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.822460 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins May 24 11:07:41.822776 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: fip_port_details.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.823689 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.824194 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: fip-port-details May 24 11:07:41.824510 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: flavors.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.825753 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.826264 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: flavors May 24 11:07:41.826586 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.828222 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.828733 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins May 24 11:07:41.829051 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: floatingip_pools.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.829753 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.830266 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: floatingip-pools May 24 11:07:41.830585 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: ip_allocation.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.831478 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.831978 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension ip_allocation not supported by any of loaded plugins May 24 11:07:41.832295 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: ip_substring_port_filtering.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.833211 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.833717 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: ip-substring-filtering May 24 11:07:41.834035 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: l2_adjacency.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.834942 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.835439 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension l2_adjacency not supported by any of loaded plugins May 24 11:07:41.835755 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: l3.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.836707 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.837239 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: router May 24 11:07:41.837559 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: l3_conntrack_helper.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.840282 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.840817 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension l3-conntrack-helper not supported by any of loaded plugins May 24 11:07:41.841178 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: l3_ext_gw_mode.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.842083 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.842597 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: ext-gw-mode May 24 11:07:41.842919 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: l3_ext_ha_mode.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.844106 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.844328 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: l3-ha May 24 11:07:41.844649 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: l3_flavors.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.845571 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.846074 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: l3-flavors May 24 11:07:41.846394 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.847289 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.847791 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: l3-port-ip-change-not-allowed May 24 11:07:41.848112 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: l3agentscheduler.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.849158 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.849672 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: l3_agent_scheduler May 24 11:07:41.849995 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: logging.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.851460 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.851968 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension logging not supported by any of loaded plugins May 24 11:07:41.852293 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: metering.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.853014 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.853616 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: metering May 24 11:07:41.853938 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.855567 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.856079 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: metering_source_and_destination_fields May 24 11:07:41.856399 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: multiprovidernet.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.857458 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.857971 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: multi-provider May 24 11:07:41.858291 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: netmtu.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.859288 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.859682 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: net-mtu May 24 11:07:41.859999 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: netmtu_writable.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.860886 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.861421 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: net-mtu-writable May 24 11:07:41.861736 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: network_availability_zone.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.862381 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.862881 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: network_availability_zone May 24 11:07:41.863195 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: network_ip_availability.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.864259 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.864766 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: network-ip-availability May 24 11:07:41.865080 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: network_segment_range.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.865844 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.866344 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension network-segment-range not supported by any of loaded plugins May 24 11:07:41.866659 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: pagination.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.867609 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.867935 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: pagination May 24 11:07:41.868248 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: port_device_profile.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.869145 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.869658 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: port-device-profile May 24 11:07:41.869974 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: port_mac_address_regenerate.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.870890 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.871400 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: port-mac-address-regenerate May 24 11:07:41.871711 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: port_numa_affinity_policy.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.872596 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.873124 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: port-numa-affinity-policy May 24 11:07:41.873447 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: port_resource_request.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.874320 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.875167 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension port-resource-request not supported by any of loaded plugins May 24 11:07:41.875167 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: portbindings.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.876018 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.876521 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: binding May 24 11:07:41.876835 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: portbindings_extended.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.878101 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.878614 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: binding-extended May 24 11:07:41.878935 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: portsecurity.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.879832 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.880342 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: port-security May 24 11:07:41.880660 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: project_id.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.881637 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.881965 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: project-id May 24 11:07:41.882283 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: providernet.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.883166 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.883677 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: provider May 24 11:07:41.883991 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: qos.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.887525 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.888044 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension qos not supported by any of loaded plugins May 24 11:07:41.888373 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: qos_bw_limit_direction.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.889610 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.890126 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension qos-bw-limit-direction not supported by any of loaded plugins May 24 11:07:41.890454 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.891344 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.891851 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins May 24 11:07:41.892169 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: qos_default.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.893046 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.893596 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension qos-default not supported by any of loaded plugins May 24 11:07:41.893897 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: qos_fip.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.894527 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.895023 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension qos-fip not supported by any of loaded plugins May 24 11:07:41.895337 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: qos_gateway_ip.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.896231 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.896738 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension qos-gateway-ip not supported by any of loaded plugins May 24 11:07:41.897058 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: qos_port_network_policy.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.897968 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.898470 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension qos-port-network-policy not supported by any of loaded plugins May 24 11:07:41.898787 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: qos_rule_type_details.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.899672 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.900174 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension qos-rule-type-details not supported by any of loaded plugins May 24 11:07:41.900496 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: qos_rules_alias.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.902054 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.902562 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension qos-rules-alias not supported by any of loaded plugins May 24 11:07:41.902881 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: quotasv2.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.903874 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per tenant" updated="2012-07-29T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.904198 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: quotas May 24 11:07:41.904512 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: quotasv2_detail.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.906133 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.906465 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: quota_details May 24 11:07:41.906791 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: rbac.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.907808 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control tenant access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.908325 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: rbac-policies May 24 11:07:41.908651 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: rbac_address_group.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.909576 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.910088 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: rbac-address-group May 24 11:07:41.910410 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: rbac_address_scope.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.911305 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.911810 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: rbac-address-scope May 24 11:07:41.912132 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: rbac_security_groups.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.913015 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.913567 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: rbac-security-groups May 24 11:07:41.913887 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: rbac_subnetpool.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.914781 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.915288 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: rbac-subnetpool May 24 11:07:41.915605 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: revisionifmatch.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.916492 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.917660 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: revision-if-match May 24 11:07:41.917972 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: revisions.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.919061 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.919573 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: standard-attr-revisions May 24 11:07:41.919891 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: router_availability_zone.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.920539 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.921048 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: router_availability_zone May 24 11:07:41.921390 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: routerservicetype.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.922278 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.922780 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension router-service-type not supported by any of loaded plugins May 24 11:07:41.923096 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.923984 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.924495 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: security-groups-normalized-cidr May 24 11:07:41.924813 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: security_groups_port_filtering.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.925776 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.926233 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: port-security-groups-filtering May 24 11:07:41.926551 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: security_groups_remote_address_group.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.927437 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.927945 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: security-groups-remote-address-group May 24 11:07:41.928267 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: securitygroup.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.938994 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.939519 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: security-group May 24 11:07:41.939847 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: segment.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.940610 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.941142 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension segment not supported by any of loaded plugins May 24 11:07:41.941467 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.942378 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.942880 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins May 24 11:07:41.943286 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: servicetype.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.944543 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.945058 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: service-type May 24 11:07:41.945589 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: sorting.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.946868 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.947195 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: sorting May 24 11:07:41.947514 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: standard_attr_segment.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.948430 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.948944 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension standard-attr-segment not supported by any of loaded plugins May 24 11:07:41.949371 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: standardattrdescription.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.949995 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.950498 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: standard-attr-description May 24 11:07:41.950813 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: stateful_security_group.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.951713 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.952225 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: stateful-security-group May 24 11:07:41.952544 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: stdattrs_common.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.953122 dv-cinder-784092-9-iscsi neutron-server[91712]: WARNING neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py May 24 11:07:41.953443 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.954401 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.954912 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension subnet-dns-publish-fixed-ip not supported by any of loaded plugins May 24 11:07:41.955231 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: subnet_onboard.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.956268 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.956780 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: subnet_onboard May 24 11:07:41.957110 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: subnet_segmentid_writable.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.958037 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.958540 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension subnet-segmentid-writable not supported by any of loaded plugins May 24 11:07:41.958859 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: subnet_service_types.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.959750 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.960262 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: subnet-service-types May 24 11:07:41.960582 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: subnetallocation.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.961763 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.962280 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: subnet_allocation May 24 11:07:41.962601 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.963245 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.964264 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: subnetpool-prefix-ops May 24 11:07:41.964591 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.965541 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.966051 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins May 24 11:07:41.966366 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: tagging.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.967402 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.967917 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: standard-attr-tag May 24 11:07:41.968238 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: timestamp.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.969447 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.969958 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loaded extension: standard-attr-timestamp May 24 11:07:41.970275 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: trunk.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.971279 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.971797 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension trunk not supported by any of loaded plugins May 24 11:07:41.972116 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: trunk_details.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.973004 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.973539 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension trunk-details not supported by any of loaded plugins May 24 11:07:41.973855 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: uplink_status_propagation.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.974753 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.975260 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension uplink-status-propagation not supported by any of loaded plugins May 24 11:07:41.975574 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Loading extension file: vlantransparent.py {{(pid=91712) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 24 11:07:41.976220 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=91712) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 24 11:07:41.976719 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extension vlan-transparent not supported by any of loaded plugins May 24 11:07:41.979169 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:network May 24 11:07:41.980451 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:subnet May 24 11:07:41.981243 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:subnetpool May 24 11:07:41.981842 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:port May 24 11:07:41.983081 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:41.983520 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service FLAVORS assigned prefix: {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:41.984124 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:41.984552 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:41.984875 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:router May 24 11:07:41.985557 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:floatingip May 24 11:07:41.986235 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service METERING assigned prefix: /metering {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:41.987672 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > rbac-policy before_create 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.988038 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > rbac-policy before_update 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.988383 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91712) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:07:41.989520 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of CountableResource for resource:rbac_policy May 24 11:07:41.989973 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:security_group May 24 11:07:41.990623 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:security_group_rule May 24 11:07:41.991337 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service CORE assigned prefix: {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:41.991674 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:subnetpool May 24 11:07:41.992003 dv-cinder-784092-9-iscsi neutron-server[91712]: WARNING neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] subnetpool is already registered May 24 11:07:41.992588 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service CORE assigned prefix: {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:41.992917 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:subnetpool May 24 11:07:41.993265 dv-cinder-784092-9-iscsi neutron-server[91712]: WARNING neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] subnetpool is already registered May 24 11:07:41.995819 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:41.996251 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service FLAVORS assigned prefix: {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:41.996836 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:41.997278 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:41.997607 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:router May 24 11:07:41.997928 dv-cinder-784092-9-iscsi neutron-server[91712]: WARNING neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] router is already registered May 24 11:07:41.998518 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:floatingip May 24 11:07:41.998841 dv-cinder-784092-9-iscsi neutron-server[91712]: WARNING neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] floatingip is already registered May 24 11:07:41.999458 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service METERING assigned prefix: /metering {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:42.000124 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of CountableResource for resource:rbac_policy May 24 11:07:42.000442 dv-cinder-784092-9-iscsi neutron-server[91712]: WARNING neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] rbac_policy is already registered May 24 11:07:42.000974 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:security_group May 24 11:07:42.001335 dv-cinder-784092-9-iscsi neutron-server[91712]: WARNING neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] security_group is already registered May 24 11:07:42.001924 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:security_group_rule May 24 11:07:42.002245 dv-cinder-784092-9-iscsi neutron-server[91712]: WARNING neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] security_group_rule is already registered May 24 11:07:42.002901 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service CORE assigned prefix: {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:42.003231 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:subnetpool May 24 11:07:42.003551 dv-cinder-784092-9-iscsi neutron-server[91712]: WARNING neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] subnetpool is already registered May 24 11:07:42.004137 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.v2.resource_helper [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Service CORE assigned prefix: {{(pid=91712) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 24 11:07:42.004458 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Creating instance of TrackedResource for resource:subnetpool May 24 11:07:42.004777 dv-cinder-784092-9-iscsi neutron-server[91712]: WARNING neutron.quota.resource_registry [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] subnetpool is already registered May 24 11:07:42.005424 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: extensions {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.006929 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: address-groups {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.008561 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: address-scopes {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.009800 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: agents {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.011520 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: auto-allocated-topology {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.012837 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: availability_zones {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.014177 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: dhcp-networks {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.015608 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: dhcp-agents {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.017043 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: routers {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.018864 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: flavors {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.020030 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: service_profiles {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.021248 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: next_providers {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.022770 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: service_profiles {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.024247 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: floatingip-pools {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.025534 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: routers {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.028094 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: floatingips {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.029244 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: l3-routers {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.030641 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: l3-agents {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.032035 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: metering-labels {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.033400 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: metering-label-rules {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.034733 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: network-ip-availabilities {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.035988 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: bindings {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.037666 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: quotas {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.039074 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: quotas {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.040471 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: rbac-policies {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.043458 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: security-groups {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.044687 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: security-group-rules {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.045963 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: service-providers {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.047195 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: subnetpools {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.159786 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: subnetpools {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.161261 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: tags {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.162979 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: tags {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.164788 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: tags {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.166592 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: tags {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.168934 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: tags {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.170861 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: tags {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.172555 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: tags {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.174460 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: tags {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.176215 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: tags {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.178097 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron.api.extensions [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Extended resource: tags {{(pid=91712) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 24 11:07:42.184637 dv-cinder-784092-9-iscsi neutron-server[91712]: WARNING keystonemiddleware.auth_token [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 24 11:07:42.189317 dv-cinder-784092-9-iscsi neutron-server[91712]: WARNING keystonemiddleware.auth_token [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 24 11:07:42.197765 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_concurrency.lockutils [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Acquired lock "singleton_lock" {{(pid=91712) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:266}} May 24 11:07:42.198264 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_concurrency.lockutils [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Releasing lock "singleton_lock" {{(pid=91712) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:282}} May 24 11:07:42.198879 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO oslo_service.service [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Starting 2 workers May 24 11:07:42.203536 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Started child 91819 {{(pid=91712) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 24 11:07:42.207873 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Started child 91820 {{(pid=91712) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 24 11:07:42.208839 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:07:42.209581 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO neutron.service [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Neutron service started, listening on 0.0.0.0:9696 May 24 11:07:42.210119 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Notify callbacks [] for process, before_spawn {{(pid=91712) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:07:42.210397 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [-] (91819) wsgi starting up on http://0.0.0.0:9696 May 24 11:07:42.212361 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:07:42.212828 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO oslo_service.service [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Starting 1 workers May 24 11:07:42.213732 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [-] (91820) wsgi starting up on http://0.0.0.0:9696 May 24 11:07:42.217644 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Started child 91821 {{(pid=91712) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 24 11:07:42.218153 dv-cinder-784092-9-iscsi neutron-server[91712]: INFO oslo_service.service [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Starting 1 workers May 24 11:07:42.221040 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.service [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] calling RpcWorker wait() {{(pid=91821) _wait /opt/stack/neutron/neutron/service.py:128}} May 24 11:07:42.221612 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.service [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] returning from RpcWorker wait() {{(pid=91821) _wait /opt/stack/neutron/neutron/service.py:135}} May 24 11:07:42.222455 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Started child 91822 {{(pid=91712) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 24 11:07:42.222925 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:07:42.223246 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG neutron_lib.callbacks.manager [None req-5c21e1bc-3309-427f-bc0c-c834d49eb70b None None] Notify callbacks [] for process, after_spawn {{(pid=91712) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:07:42.223418 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [-] neutron.plugins.ml2.plugin.Ml2Plugin method start_rpc_listeners called with arguments () {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:07:42.225195 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] Full set of CONF: {{(pid=91712) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:650}} May 24 11:07:42.225415 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 11:07:42.225597 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] Configuration options gathered from: {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 11:07:42.225811 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 11:07:42.226145 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 11:07:42.226309 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ================================================================================ {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 11:07:42.226826 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] agent_down_time = 75 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.227091 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] allow_automatic_dhcp_failover = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.227373 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] allow_automatic_l3agent_failover = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.227641 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] allow_bulk = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.227944 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] allow_overlapping_ips = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.228046 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:07:42.228526 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.228965 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] api_extensions_path = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.229245 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] api_paste_config = api-paste.ini {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.230062 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] api_workers = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.230323 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] auth_strategy = keystone {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.230491 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] backdoor_port = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.230658 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] backdoor_socket = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.230902 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] backlog = 4096 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.231150 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] base_mac = fa:16:3e:00:00:00 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.231467 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] bind_host = 0.0.0.0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.231718 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] bind_port = 9696 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.231962 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] client_socket_timeout = 900 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.232221 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] config_dir = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.232398 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.232667 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] config_source = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.232945 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] control_exchange = neutron {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.233197 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] core_plugin = ml2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.233470 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] debug = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.233742 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] default_availability_zones = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.234531 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.234531 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] dhcp_agent_notification = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.234825 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] dhcp_agents_per_network = 1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.235126 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] dhcp_lease_duration = 86400 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.235264 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] dhcp_load_type = networks {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.235388 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] dns_domain = openstacklocal {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.235631 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.235879 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] enable_dvr = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.236109 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] enable_new_agents = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.236352 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] enable_services_on_agents_with_admin_state_down = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.236613 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] enable_snat_by_default = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.236868 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] enable_traditional_dhcp = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.237126 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] external_dns_driver = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.237372 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] filter_validation = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.237617 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] global_physnet_mtu = 1500 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.237832 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] graceful_shutdown_timeout = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.238316 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] host = dv-cinder-784092-9-iscsi {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.238531 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] host_dvr_for_dhcp = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.238761 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] http_retries = 3 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.239002 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] instance_format = [instance: %(uuid)s] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.239255 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.239495 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ipam_driver = internal {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.239707 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ipv6_pd_enabled = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.239965 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] l3_ha = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.240220 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.240482 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] l3_ha_network_physical_name = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.240712 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] l3_ha_network_type = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.241018 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_config_append = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.241193 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.241405 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_dir = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.241641 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_file = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.241856 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_options = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.242009 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_rotate_interval = 1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.242289 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_rotate_interval_type = days {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.242584 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_rotation_type = none {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.242745 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.242965 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.243108 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.243376 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.243519 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.243729 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_allowed_address_pair = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.243961 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_dns_nameservers = 5 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.244177 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_header_line = 16384 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.244391 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_l3_agents_per_router = 3 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.244632 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_logfile_count = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.244868 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_logfile_size_mb = 200 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.245723 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_routes = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.245982 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_subnet_host_routes = 20 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.246223 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] network_auto_schedule = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.246433 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] network_link_prefix = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.246676 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.246910 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] notify_nova_on_port_data_changes = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.247154 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] notify_nova_on_port_status_changes = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.247399 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] pagination_max_limit = -1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.247622 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] periodic_fuzzy_delay = 5 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.247839 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] periodic_interval = 40 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.248052 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] publish_errors = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.248272 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rate_limit_burst = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.248491 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rate_limit_except_level = CRITICAL {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.248723 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rate_limit_interval = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.248937 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] retry_until_window = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.249212 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] router_auto_schedule = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.249430 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] router_distributed = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.249686 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.LeastRoutersScheduler {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.249936 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rpc_response_max_timeout = 600 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.250204 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rpc_response_timeout = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.251061 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rpc_state_report_workers = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.251061 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rpc_workers = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.251061 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] send_events_interval = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.251330 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.251782 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] setproctitle = on {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.252393 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] state_path = /opt/stack/data/neutron {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.252849 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] syslog_log_facility = LOG_USER {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.253039 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] tcp_keepidle = 600 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.253346 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] transport_url = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.253535 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] use_eventlog = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.253855 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] use_journal = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.254043 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] use_json = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.254282 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] use_ssl = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.254817 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] use_stderr = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.254817 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] use_syslog = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.255095 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] vlan_transparent = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.255095 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] watch_log_file = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.255853 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] wsgi_default_pool_size = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.255853 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] wsgi_keep_alive = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.255853 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.257276 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_concurrency.disable_process_locking = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.257276 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.257276 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.connection_string = messaging:// {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.257276 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.enabled = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.257637 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.es_doc_type = notification {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.258068 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.es_scroll_size = 10000 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.258591 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.es_scroll_time = 2m {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.258918 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.services.l3_router.l3_router_plugin [-] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method start_rpc_listeners called with arguments () {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:07:42.259128 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.filter_error_trace = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.259402 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.hmac_keys = SECRET_KEY {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.259654 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.sentinel_service_name = mymaster {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.259985 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.socket_timeout = 0.1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.260240 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.trace_sqlalchemy = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.260526 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.enforce_new_defaults = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.261574 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.enforce_scope = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.261574 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.policy_default_rule = default {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.261574 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.261574 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.261574 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.261574 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.261574 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.261574 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.265489 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.265489 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.auth_section = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.265489 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.auth_type = password {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.265489 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.auth_url = http://10.222.0.18/identity {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.265489 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.265489 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.265489 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.collect_timing = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.265489 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.default_domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.266025 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.default_domain_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.266025 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.266025 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.domain_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.266025 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.endpoint_type = public {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.266025 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.insecure = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.266376 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.266376 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.live_migration_events = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.266610 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.password = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.266834 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.project_domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.267068 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.project_domain_name = Default {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.268500 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.project_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.268500 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.project_name = service {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.268500 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.region_name = RegionOne {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.268500 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.split_loggers = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.268500 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.system_scope = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.268500 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.269035 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.trust_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.269035 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.user_domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.269238 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.user_domain_name = Default {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.269329 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.user_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.269565 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.username = nova {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.269797 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.auth_section = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.270025 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.auth_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.273411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.cafile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.273411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.273411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.collect_timing = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.273411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.endpoint_type = public {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.273411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.insecure = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.273411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.273411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.region_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.273411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.split_loggers = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.273411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.276587 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.auth_section = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.276587 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.auth_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.276587 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.cafile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.276587 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.276587 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.collect_timing = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.276587 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.connect_retries = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.276587 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.connect_retry_delay = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.276587 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.enable_notifications = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.276587 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.endpoint_override = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.278626 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.insecure = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.278626 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.interface = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.278626 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.278626 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.max_version = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.278626 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.min_version = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.278626 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.region_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.278626 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.service_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.278626 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.service_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.278626 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.split_loggers = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.279449 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.status_code_retries = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.279449 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.status_code_retry_delay = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.279449 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.279449 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.valid_interfaces = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.279449 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.version = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.279449 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] service_providers.service_provider = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.279449 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.279449 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.280013 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.280013 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.280013 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.280013 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.280013 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.280013 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.280013 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.280013 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.280539 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.280539 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.280679 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.282037 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.282037 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.282037 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.282037 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.282767 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.282767 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.283018 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.283143 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.283399 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.283584 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.283911 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.284277 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.284407 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_version = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.285978 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_notifications.driver = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.285978 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_notifications.retry = -1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.285978 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.285978 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_notifications.transport_url = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.285978 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.285978 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.default_quota = -1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.286899 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_driver = neutron.db.quota.driver.DbQuotaDriver {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.286899 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_floatingip = 50 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.286899 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_network = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.287251 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_port = 500 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.287251 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_rbac_policy = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.287627 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_router = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.287627 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.288149 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group_rule = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.288149 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_subnet = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.288402 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.track_quota_usage = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.289370 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_middleware.enable_proxy_headers_parsing = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.289370 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] healthcheck.backends = ['disable_by_file'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.289370 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] healthcheck.detailed = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.289370 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] healthcheck.disable_by_file_path = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.289890 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] healthcheck.path = /healthcheck {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.289890 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.admin_password = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.290144 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.290276 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.admin_token = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.290651 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.admin_user = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.290796 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_admin_prefix = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.290992 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.294085 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_port = 35357 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.294085 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_protocol = https {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.294085 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_section = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.294085 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_type = password {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.294085 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_uri = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.294085 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_url = http://10.222.0.18/identity {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.294085 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_version = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.294085 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.cache = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.295769 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.295769 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.295769 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.295769 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.295769 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.delay_auth_decision = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.295769 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.295769 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.domain_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.295769 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.297945 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.http_connect_timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.297945 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.297945 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.identity_uri = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.297945 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.include_service_catalog = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.297945 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.insecure = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.297945 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.interface = public {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.297945 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.297945 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.298509 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.298509 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.298509 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.298509 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.298509 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.298941 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.298941 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_use_advanced_pool = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.299098 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.299318 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.password = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.299550 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.299780 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_name = Default {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.300018 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.project_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.300329 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.project_name = service {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.300553 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.region_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.300956 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.301202 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles_required = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.301413 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.service_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.301625 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.system_scope = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.301862 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.token_cache_time = 300 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.302081 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.trust_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.302322 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.302577 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_name = Default {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.302795 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.user_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.303047 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.username = neutron {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.303298 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.303624 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.backend = dogpile.cache.null {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.303871 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.backend_argument = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.304096 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.config_prefix = cache.oslo {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.304330 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.debug_cache_backend = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.304545 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.enabled = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.304779 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.expiration_time = 600 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.305038 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_dead_retry = 300 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.305290 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.305513 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.305750 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_pool_maxsize = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.305986 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.306241 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.306484 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_socket_timeout = 1.0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.306707 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.proxies = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.306935 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.tls_allowed_ciphers = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.307144 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.tls_cafile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.307352 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.tls_certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.307643 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.tls_enabled = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.308362 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.tls_keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.309128 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.309128 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep.group = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.309387 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep.helper_command = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.309387 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep.thread_pool_size = 4 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.309644 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep.user = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.310207 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.310207 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_dhcp_release.group = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.310466 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_dhcp_release.helper_command = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.310466 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_dhcp_release.thread_pool_size = 4 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.310709 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_dhcp_release.user = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.310885 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.311093 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.group = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.311304 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.helper_command = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.311525 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.thread_pool_size = 4 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.311756 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.user = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.311979 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_namespace.capabilities = [21] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.312189 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_namespace.group = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.312398 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_namespace.helper_command = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.312618 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_namespace.thread_pool_size = 4 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.312826 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_namespace.user = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.314153 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_conntrack.capabilities = [12] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.314153 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_conntrack.group = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.314153 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_conntrack.helper_command = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.314153 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_conntrack.thread_pool_size = 4 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.314153 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_conntrack.user = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.314505 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] AGENT.root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.314505 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] AGENT.root_helper_daemon = sudo /usr/local/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.314662 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] AGENT.use_helper_for_ns_read = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.315097 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_ipset = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.315353 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_security_group = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.315596 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] SECURITYGROUP.firewall_driver = openvswitch {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.315850 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.316170 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.extension_drivers = ['port_security'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.316659 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.external_network_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.316659 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.mechanism_drivers = ['openvswitch', 'hyperv'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.316889 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.overlay_ip_version = 4 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.317285 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.path_mtu = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.317285 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.physical_network_mtus = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.317551 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.tenant_network_types = ['vlan'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.317889 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.type_drivers = ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.318188 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_flat.flat_networks = ['physnet1'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.318443 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_geneve.max_header_size = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.318668 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_geneve.vni_ranges = ['1:1000'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.318962 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_gre.tunnel_id_ranges = ['1:1000'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.319247 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_vlan.network_vlan_ranges = ['physnet1:500:549'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.319523 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_vxlan.vni_ranges = ['1:1000'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.319737 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_vxlan.vxlan_group = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.320028 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] OVS_DRIVER.vnic_type_prohibit_list = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.320366 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.backend = sqlalchemy {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.320643 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.connection = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.320897 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.connection_debug = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.321173 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.connection_parameters = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.321699 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.connection_recycle_time = 3600 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.321925 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.connection_trace = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.322140 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.db_inc_retry_interval = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.322374 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.db_max_retries = 20 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.322622 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.db_max_retry_interval = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.322853 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.db_retry_interval = 1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.323095 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.max_overflow = 50 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.323347 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.max_pool_size = 50 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.323591 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.max_retries = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.323823 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.mysql_enable_ndb = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.324066 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.324314 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.pool_timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.324557 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.retry_interval = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.324767 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.slave_connection = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.325008 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.sqlite_synchronous = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.325249 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.use_db_reconnect = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.325405 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 11:07:42.325960 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] Full set of CONF: {{(pid=91712) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:650}} May 24 11:07:42.326120 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 11:07:42.326282 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] Configuration options gathered from: {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 11:07:42.326469 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 11:07:42.326638 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 11:07:42.326788 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ================================================================================ {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 11:07:42.326990 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] agent_down_time = 75 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.327150 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] allow_automatic_dhcp_failover = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.327309 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] allow_automatic_l3agent_failover = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.327467 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] allow_bulk = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.327625 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] allow_overlapping_ips = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.327825 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.328014 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] api_extensions_path = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.328173 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] api_paste_config = api-paste.ini {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.328330 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] api_workers = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.328488 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] auth_strategy = keystone {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.328645 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] backdoor_port = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.328801 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] backdoor_socket = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.328956 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] backlog = 4096 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.329135 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] base_mac = fa:16:3e:00:00:00 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.329292 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] bind_host = 0.0.0.0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.329604 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] bind_port = 9696 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.329764 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] client_socket_timeout = 900 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.329923 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] config_dir = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.330085 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.330243 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] config_source = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.330400 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] control_exchange = neutron {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.330556 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] core_plugin = ml2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.330712 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] debug = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.330872 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] default_availability_zones = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.331045 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.331205 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] dhcp_agent_notification = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.331361 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] dhcp_agents_per_network = 1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.331517 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] dhcp_lease_duration = 86400 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.331677 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] dhcp_load_type = networks {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.331879 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] dns_domain = openstacklocal {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.332043 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.332245 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] enable_dvr = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.332408 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] enable_new_agents = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.332609 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] enable_services_on_agents_with_admin_state_down = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.332851 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] enable_snat_by_default = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.333050 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] enable_traditional_dhcp = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.333324 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] external_dns_driver = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.333569 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] filter_validation = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.333778 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] global_physnet_mtu = 1500 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.334035 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] graceful_shutdown_timeout = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.335250 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] host = dv-cinder-784092-9-iscsi {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.335411 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] host_dvr_for_dhcp = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.335568 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] http_retries = 3 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.335709 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] instance_format = [instance: %(uuid)s] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.335855 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.335995 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ipam_driver = internal {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.336349 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ipv6_pd_enabled = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.336349 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] l3_ha = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.336624 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.336624 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] l3_ha_network_physical_name = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.336882 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] l3_ha_network_type = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.336882 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_config_append = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.337195 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.337195 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_dir = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.337458 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_file = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.337458 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_options = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.337713 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_rotate_interval = 1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.337869 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_rotate_interval_type = days {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.337869 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] log_rotation_type = none {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.338147 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.338264 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.338392 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.338527 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.338653 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.338799 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_allowed_address_pair = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.338933 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_dns_nameservers = 5 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.339090 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_header_line = 16384 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.339247 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_l3_agents_per_router = 3 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.339402 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_logfile_count = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.339557 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_logfile_size_mb = 200 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.339713 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_routes = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.340120 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] max_subnet_host_routes = 20 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.340120 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] network_auto_schedule = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.340384 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] network_link_prefix = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.340384 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.340626 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] notify_nova_on_port_data_changes = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.340626 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] notify_nova_on_port_status_changes = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.340856 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] pagination_max_limit = -1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.340856 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] periodic_fuzzy_delay = 5 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.341117 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] periodic_interval = 40 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.341257 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] publish_errors = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.341341 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rate_limit_burst = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.341495 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rate_limit_except_level = CRITICAL {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.341650 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rate_limit_interval = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.341806 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] retry_until_window = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.341962 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] router_auto_schedule = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.342118 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] router_distributed = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.342279 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.LeastRoutersScheduler {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.342404 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rpc_response_max_timeout = 600 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.342564 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rpc_response_timeout = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.342720 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rpc_state_report_workers = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.342876 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] rpc_workers = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.343031 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] send_events_interval = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.343196 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.343353 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] setproctitle = on {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.343553 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] state_path = /opt/stack/data/neutron {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.344016 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] syslog_log_facility = LOG_USER {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.344016 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] tcp_keepidle = 600 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.344016 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] transport_url = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.344624 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] use_eventlog = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.344624 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] use_journal = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.344624 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] use_json = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.345060 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] use_ssl = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.345293 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] use_stderr = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.345424 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] use_syslog = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.345567 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] vlan_transparent = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.345710 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] watch_log_file = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.345931 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] wsgi_default_pool_size = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.345931 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] wsgi_keep_alive = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.346182 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 11:07:42.346306 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_concurrency.disable_process_locking = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.346495 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.346622 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.connection_string = messaging:// {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.346744 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.enabled = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.346866 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.es_doc_type = notification {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.347128 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.es_scroll_size = 10000 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.347128 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.es_scroll_time = 2m {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.347378 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.filter_error_trace = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.347502 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.hmac_keys = SECRET_KEY {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.347625 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.sentinel_service_name = mymaster {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.347747 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.socket_timeout = 0.1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.347896 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] profiler.trace_sqlalchemy = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.348250 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.enforce_new_defaults = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.348250 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.enforce_scope = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.348495 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.policy_default_rule = default {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.348627 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.348750 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.348879 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.349003 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.349145 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.349270 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.349434 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.349610 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.auth_section = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.349769 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.auth_type = password {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.349929 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.auth_url = http://10.222.0.18/identity {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.350087 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.350247 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.350403 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.collect_timing = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.350554 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.default_domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.350713 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.default_domain_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.350873 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.351053 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.domain_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.351173 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.services.l3_router.l3_router_plugin [-] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method start_rpc_listeners called with arguments () {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:07:42.351275 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.endpoint_type = public {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.351357 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.insecure = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.351486 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.351644 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.live_migration_events = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.351803 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.password = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.351961 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.project_domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.352120 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.project_domain_name = Default {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.352280 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.project_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.352439 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.project_name = service {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.352597 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.region_name = RegionOne {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.352756 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.split_loggers = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.352916 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.system_scope = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.353074 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.353258 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.trust_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.353416 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.user_domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.353574 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.user_domain_name = Default {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.353733 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.user_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.353891 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] nova.username = nova {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.354059 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.auth_section = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.354220 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.auth_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.354881 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.cafile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.355045 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.355205 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.collect_timing = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.355363 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.endpoint_type = public {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.355519 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.insecure = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.355677 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.355835 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.region_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.355993 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.split_loggers = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.356151 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] placement.timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.356322 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.auth_section = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.356480 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.auth_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.356637 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.cafile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.356795 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.356954 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.collect_timing = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.357133 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.connect_retries = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.357296 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.connect_retry_delay = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.357453 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.enable_notifications = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.357612 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.endpoint_override = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.357769 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.insecure = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.357928 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.interface = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.358085 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.358243 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.max_version = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.358402 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.min_version = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.358560 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.region_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.358718 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.service_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.358876 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.service_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.359034 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.split_loggers = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.359191 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.status_code_retries = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.359348 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.status_code_retry_delay = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.359507 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.359664 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.valid_interfaces = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.359822 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ironic.version = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.359988 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] service_providers.service_provider = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.360160 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.360320 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.360477 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.360636 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.360795 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.360953 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.361127 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.361289 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.361447 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.361605 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.361762 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.361919 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.362080 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.362238 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.362396 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.362553 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.362711 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.362868 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.363025 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.363182 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.363340 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.363553 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.363649 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.363816 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.364169 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.364423 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_version = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.364598 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_notifications.driver = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.364758 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_notifications.retry = -1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.364898 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.365450 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_messaging_notifications.transport_url = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.365450 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.365450 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.default_quota = -1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.366101 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_driver = neutron.db.quota.driver.DbQuotaDriver {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.366101 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_floatingip = 50 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.366101 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_network = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.366719 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_port = 500 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.366719 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_rbac_policy = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.366719 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_router = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.366719 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.367329 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group_rule = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.367329 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.quota_subnet = 100 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.367329 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] QUOTAS.track_quota_usage = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.367329 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] oslo_middleware.enable_proxy_headers_parsing = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.367841 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] healthcheck.backends = ['disable_by_file'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.367841 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] healthcheck.detailed = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.367841 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] healthcheck.disable_by_file_path = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.367841 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] healthcheck.path = /healthcheck {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.368560 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.admin_password = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.368560 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.368560 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.admin_token = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.368560 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.admin_user = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.368972 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_admin_prefix = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.368972 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.368972 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_port = 35357 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.369343 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_protocol = https {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.369343 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_section = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.369343 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_type = password {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.369555 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_uri = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.369655 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_url = http://10.222.0.18/identity {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.369815 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.auth_version = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.369975 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.cache = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.370742 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.370902 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.371063 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.371222 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.371381 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.delay_auth_decision = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.371541 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.371701 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.domain_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.371859 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.372017 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.http_connect_timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.372176 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.372334 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.identity_uri = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.372493 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.include_service_catalog = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.372652 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.insecure = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.372811 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.interface = public {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.372969 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.373149 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.373310 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.373469 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.373628 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.373790 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.373950 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.374109 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.374266 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_use_advanced_pool = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.374495 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.374761 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.password = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.374761 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.374995 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_name = Default {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.375209 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.project_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.375209 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.project_name = service {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.375544 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.region_name = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.375544 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.375695 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles_required = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.375771 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.service_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.375915 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.system_scope = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.376072 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.token_cache_time = 300 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.376231 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.trust_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.376389 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.376548 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_name = Default {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.376707 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.user_id = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.376866 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.username = neutron {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.377024 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.377218 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.backend = dogpile.cache.null {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.377430 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.backend_argument = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.377559 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.config_prefix = cache.oslo {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.377682 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.debug_cache_backend = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.377803 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.enabled = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.378131 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.expiration_time = 600 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.378131 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_dead_retry = 300 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.378386 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.378386 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.378582 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_pool_maxsize = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.378702 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.378848 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.379011 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.memcache_socket_timeout = 1.0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.379169 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.proxies = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.379327 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.tls_allowed_ciphers = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.379486 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.tls_cafile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.379644 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.tls_certfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.379803 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.tls_enabled = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.379961 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] cache.tls_keyfile = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.380132 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.380290 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep.group = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.380449 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep.helper_command = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.380607 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep.thread_pool_size = 4 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.380766 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep.user = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.380934 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.381110 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_dhcp_release.group = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.381272 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_dhcp_release.helper_command = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.381429 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_dhcp_release.thread_pool_size = 4 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.381587 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_dhcp_release.user = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.381756 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.381914 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.group = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.382073 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.helper_command = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.382231 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.thread_pool_size = 4 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.382389 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.user = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.382556 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_namespace.capabilities = [21] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.382714 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_namespace.group = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.382872 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_namespace.helper_command = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.383031 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_namespace.thread_pool_size = 4 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.383189 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_namespace.user = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.383355 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_conntrack.capabilities = [12] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.383514 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_conntrack.group = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.383673 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_conntrack.helper_command = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.383835 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_conntrack.thread_pool_size = 4 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.383992 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] privsep_conntrack.user = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.384157 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] AGENT.root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.384316 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] AGENT.root_helper_daemon = sudo /usr/local/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.384474 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] AGENT.use_helper_for_ns_read = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.384645 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_ipset = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.384787 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_security_group = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.384945 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] SECURITYGROUP.firewall_driver = openvswitch {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.385125 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.385300 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.extension_drivers = ['port_security'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.385459 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.external_network_type = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.385639 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.mechanism_drivers = ['openvswitch', 'hyperv'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.385799 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.overlay_ip_version = 4 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.385957 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.path_mtu = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.386116 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.physical_network_mtus = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.386276 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.tenant_network_types = ['vlan'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.386439 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2.type_drivers = ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.386605 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_flat.flat_networks = ['physnet1'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.386762 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_geneve.max_header_size = 30 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.386922 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_geneve.vni_ranges = ['1:1000'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.387089 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_gre.tunnel_id_ranges = ['1:1000'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.387256 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_vlan.network_vlan_ranges = ['physnet1:500:549'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.387424 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_vxlan.vni_ranges = ['1:1000'] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.387572 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ml2_type_vxlan.vxlan_group = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.387736 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] OVS_DRIVER.vnic_type_prohibit_list = [] {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.387906 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.backend = sqlalchemy {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.388065 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.connection = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.388223 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.connection_debug = 0 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.388381 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.connection_parameters = {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.388538 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.connection_recycle_time = 3600 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.388696 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.connection_trace = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.388855 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.db_inc_retry_interval = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.389013 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.db_max_retries = 20 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.389199 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.db_max_retry_interval = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.389359 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.db_retry_interval = 1 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.389517 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.max_overflow = 50 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.389675 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.max_pool_size = 50 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.389834 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.max_retries = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.389992 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.mysql_enable_ndb = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.390151 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.390348 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.pool_timeout = None {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.390457 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.retry_interval = 10 {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.390614 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.slave_connection = **** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.390772 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.sqlite_synchronous = True {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.390931 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] database.use_db_reconnect = False {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 11:07:42.391081 dv-cinder-784092-9-iscsi neutron-server[91712]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=91712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 11:07:42.521850 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 49880) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:07:42.530762 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [-] 10.222.0.18 "GET / HTTP/1.1" status: 200 len: 253 time: 0.0067437 May 24 11:07:53.437680 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-47820c6e-215f-4528-bc53-d5a0547a2ed8 None None] get_active_networks_info from dv-cinder-784092-9-iscsi {{(pid=91821) get_active_networks_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:158}} May 24 11:07:53.503182 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-47820c6e-215f-4528-bc53-d5a0547a2ed8 None None] No non-hosted networks {{(pid=91821) auto_schedule_networks /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:60}} May 24 11:07:53.518402 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.agentschedulers_db [None req-47820c6e-215f-4528-bc53-d5a0547a2ed8 None None] DHCP Agent not found on host dv-cinder-784092-9-iscsi {{(pid=91821) list_active_networks_on_active_dhcp_agent /opt/stack/neutron/neutron/db/agentschedulers_db.py:455}} May 24 11:07:53.616075 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e8b64784-d16b-47d2-a8ea-ef033dbeeb50 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703'] for agent, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:07:55.766093 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703'] for agent, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:07:55.768540 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:07:55.769919 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:07:55.771106 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:07:55.772634 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:07:55.775052 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:07:55.775628 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:07:55.778644 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:07:55.779751 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:07:55.781227 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:07:55.783398 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:07:55.784656 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:07:55.785332 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:07:55.786528 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] neutron.api.rpc.handlers.resources_rpc.ResourcesPushToServersRpcApi method report_agent_resource_versions called with arguments (, 'Open vSwitch agent', 'dv-cinder-784092-9-iscsi', {'QosPolicy': '1.8', 'Trunk': '1.1', 'SubPort': '1.0', 'Port': '1.7', 'Subnet': '1.1', 'Network': '1.1', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'Log': '1.0', 'PortForwarding': '1.2', 'ConntrackHelper': '1.0', 'AddressGroup': '1.2'}) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:07:55.800958 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] neutron.api.rpc.handlers.resources_rpc.ResourcesPushToServerRpcCallback method report_agent_resource_versions called with arguments (,) {'agent_type': 'Open vSwitch agent', 'agent_host': 'dv-cinder-784092-9-iscsi', 'version_map': {'QosPolicy': '1.8', 'Trunk': '1.1', 'SubPort': '1.0', 'Port': '1.7', 'Subnet': '1.1', 'Network': '1.1', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'Log': '1.0', 'PortForwarding': '1.2', 'ConntrackHelper': '1.0', 'AddressGroup': '1.2'}} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:07:55.843441 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-557c35db-1981-4ac0-bbff-ac94ba6a0599 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703'] for agent, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:07:55.930540 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a9b22e5b-eaeb-443b-94e1-e792b3a77289 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:07:57.016689 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703'] for agent, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:07:58.866553 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703'] for agent, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:07:59.074416 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.db.l3_hamode_db [None req-63dfd75a-33a0-4e07-bc92-9cb90f52c8d0 None None] Number of active agents lower than max_l3_agents_per_router. L3 agents available: 1 May 24 11:07:59.170272 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-63dfd75a-33a0-4e07-bc92-9cb90f52c8d0 None None] Routers set() bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:07:59.170659 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-63dfd75a-33a0-4e07-bc92-9cb90f52c8d0 None None] Router IDs set() for agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:464}} May 24 11:08:00.602014 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 49970) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:01.191837 dv-cinder-784092-9-iscsi neutron-server[91819]: WARNING oslo_policy.policy [None req-e6440209-37da-48ab-a86f-a228f5953c42 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 11:08:01.192406 dv-cinder-784092-9-iscsi neutron-server[91819]: WARNING oslo_policy.policy [None req-e6440209-37da-48ab-a86f-a228f5953c42 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 11:08:01.408476 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [None req-e6440209-37da-48ab-a86f-a228f5953c42 admin admin] http://10.222.0.18:9696/v2.0/extensions returned with HTTP 200 {{(pid=91819) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 11:08:01.409726 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e6440209-37da-48ab-a86f-a228f5953c42 admin admin] 10.222.0.18 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.8057230 May 24 11:08:03.737967 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 49984) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:04.355975 dv-cinder-784092-9-iscsi neutron-server[91820]: WARNING oslo_policy.policy [None req-270dea7d-4c82-48df-8679-c0f4dd3f66e7 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 11:08:04.356478 dv-cinder-784092-9-iscsi neutron-server[91820]: WARNING oslo_policy.policy [None req-270dea7d-4c82-48df-8679-c0f4dd3f66e7 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 11:08:04.573620 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-270dea7d-4c82-48df-8679-c0f4dd3f66e7 admin admin] Request body: {'subnetpool': {'shared': True, 'default_prefixlen': 26, 'is_default': True, 'name': 'shared-default-subnetpool-v4', 'prefixes': ['172.20.0.0/16']}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:08:04.674038 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.policy [None req-270dea7d-4c82-48df-8679-c0f4dd3f66e7 admin admin] Loaded default policies from ['neutron'] under neutron.policies entry points {{(pid=91820) register_rules /opt/stack/neutron/neutron/policy.py:73}} May 24 11:08:04.719868 dv-cinder-784092-9-iscsi neutron-server[91820]: WARNING oslo_policy.policy [None req-270dea7d-4c82-48df-8679-c0f4dd3f66e7 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 11:08:04.720236 dv-cinder-784092-9-iscsi neutron-server[91820]: WARNING oslo_policy.policy [None req-270dea7d-4c82-48df-8679-c0f4dd3f66e7 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 11:08:04.729414 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.quota [None req-270dea7d-4c82-48df-8679-c0f4dd3f66e7 admin admin] Loaded quota_driver: . May 24 11:08:04.745140 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-270dea7d-4c82-48df-8679-c0f4dd3f66e7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:08:04.817697 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-270dea7d-4c82-48df-8679-c0f4dd3f66e7 admin admin] Notify callbacks ['neutron.objects.securitygroup.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8779720905073', 'neutron.objects.address_group.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8779720905510', 'neutron.objects.network.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8779720887743', 'neutron.objects.qos.policy.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8779720888531', 'neutron.objects.subnetpool.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8779720936431', 'neutron.objects.address_scope.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8779720938759', 'neutron.plugins.ml2.plugin.External_net_db_mixin._process_ext_policy_create-1741581', 'neutron.plugins.ml2.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-1876111', 'neutron.services.network_ip_availability.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-1011314', 'neutron.services.network_ip_availability.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-918539'] for rbac-policy, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:04.897678 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-270dea7d-4c82-48df-8679-c0f4dd3f66e7 admin admin] Notify callbacks [] for subnetpool, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:04.899407 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-270dea7d-4c82-48df-8679-c0f4dd3f66e7 admin admin] 10.222.0.18 "POST /v2.0/subnetpools HTTP/1.1" status: 201 len: 739 time: 1.1603818 May 24 11:08:07.557135 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50002) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:07.911300 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Request body: {'network': {'project_id': '25622105db07401c9ad4153c1d775107', 'admin_state_up': True, 'tenant_id': '25622105db07401c9ad4153c1d775107', 'name': 'private'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:08:07.916051 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:08:07.920528 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Usage tracker for resource:network and tenant:25622105db07401c9ad4153c1d775107 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:08:07.925035 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:08:07.931132 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:08:07.941458 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Made reservation on behalf of 25622105db07401c9ad4153c1d775107 for: {'network': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:08:07.944190 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:07.962245 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:07.976210 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:08:07.979858 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Usage tracker for resource:security_group_rule and tenant:25622105db07401c9ad4153c1d775107 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:08:07.984883 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:08:07.990495 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:08:08.088915 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:08.093234 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:08.097785 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:08.116804 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] vlan segment allocate from pool success with {'vlan_id': 535, 'physical_network': 'physnet1'} {{(pid=91820) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:08:08.143331 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Notify callbacks ['neutron.services.segments.db._add_segment_host_mapping_for_segment-8779719283694', 'neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:08.143942 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Unsubscribe: segment precommit_create {{(pid=91820) unsubscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:83}} May 24 11:08:08.144407 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.segments_db [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Added segment 0e9fee1d-025f-4262-885a-f1e7a3dccf62 of type vlan for network 77df64b5-a17d-48ec-a320-103003694065 May 24 11:08:08.192108 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=92b28f60-0b86-4728-9a75-539f663d2104,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:08.214619 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:08.214956 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:08.215278 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:08.215595 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:08.215908 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:08.216218 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:08.216525 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:08.216830 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:08.217157 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:08.217461 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:08.217761 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:08.218060 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Tracked resource versions report: May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: pushed versions: May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: { 'AddressGroup': {'1.2'}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'ConntrackHelper': {'1.0'}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Log': {'1.0'}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Network': {'1.1'}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Port': {'1.7'}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'PortForwarding': {'1.2'}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'QosPolicy': {'1.8'}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'SecurityGroup': {'1.4'}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'SecurityGroupRule': {'1.2'}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'SubPort': {'1.0'}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Subnet': {'1.1'}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Trunk': {'1.1'}} May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: consumer versions: May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: { DHCP agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: L3 agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: Metadata agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: Metering agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: Open vSwitch agent@dv-cinder-784092-9-iscsi: { 'AddressGroup': '1.2', May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'ConntrackHelper': '1.0', May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Log': '1.0', May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Network': '1.1', May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Port': '1.7', May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'PortForwarding': '1.2', May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'QosPolicy': '1.8', May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'SecurityGroup': '1.4', May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'SecurityGroupRule': '1.2', May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'SubPort': '1.0', May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Subnet': '1.1', May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Trunk': '1.1'}} May 24 11:08:08.219138 dv-cinder-784092-9-iscsi neutron-server[91820]: {{(pid=91820) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 11:08:08.262414 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:08.273876 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.176s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:08.315505 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:08.317003 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:08:08.317646 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:08.320926 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:08.424997 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 849 time: 0.8670111 May 24 11:08:08.425667 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Pushing event updated for resources: {'Network': ['ID=77df64b5-a17d-48ec-a320-103003694065,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:08.429001 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:08.439816 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network 77df64b5-a17d-48ec-a320-103003694065 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:08:11.147886 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50018) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:11.607863 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c92f1bb5-86f5-431e-9fb8-1556a497fa3f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:08:11.609267 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-c92f1bb5-86f5-431e-9fb8-1556a497fa3f admin admin] 10.222.0.18 "GET /v2.0/networks/77df64b5-a17d-48ec-a320-103003694065 HTTP/1.1" status: 200 len: 825 time: 0.4603879 May 24 11:08:11.636958 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-418e5ddc-e200-4157-a7d4-f53f8cb5cd0c admin admin] 10.222.0.18 "GET /v2.0/subnetpools/ce781e3a-a7da-4efc-9d7e-994a282506de HTTP/1.1" status: 200 len: 734 time: 0.0220613 May 24 11:08:11.645638 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Request body: {'subnet': {'name': 'private-subnet', 'subnetpool_id': 'ce781e3a-a7da-4efc-9d7e-994a282506de', 'ip_version': 4, 'network_id': '77df64b5-a17d-48ec-a320-103003694065', 'gateway_ip': '172.20.0.1', 'tenant_id': '25622105db07401c9ad4153c1d775107'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:08:11.650280 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:08:11.654570 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Usage tracker for resource:subnet and tenant:25622105db07401c9ad4153c1d775107 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:08:11.657414 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:08:11.664288 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:08:11.674629 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Made reservation on behalf of 25622105db07401c9ad4153c1d775107 for: {'subnet': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:08:11.677025 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:12.164793 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781'] for subnet, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:12.168863 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:12.251911 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Pushing event updated for resources: {'Subnet': ['ID=3d2ff612-067b-478d-bf96-a25c339bd70d,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:12.256314 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Checking segment: {'id': '0e9fee1d-025f-4262-885a-f1e7a3dccf62', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 535, 'network_id': '77df64b5-a17d-48ec-a320-103003694065'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:08:12.256698 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Network hostable DHCP agents. Network: 77df64b5-a17d-48ec-a320-103003694065, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91820) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:08:12.259665 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:12.278026 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Network 77df64b5-a17d-48ec-a320-103003694065 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91820) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:08:12.288865 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, 77df64b5-a17d-48ec-a320-103003694065) {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:08:12.294396 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:08:12.295013 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:12.299427 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-977496cc-a4f8-4114-b544-a5f8d7f4cc48 None None] Network 77df64b5-a17d-48ec-a320-103003694065 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:08:12.320107 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 854 time: 0.6777225 May 24 11:08:12.624450 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpea482e27-f15a-5756-a111-7e0dba035002-77df64b5-a17d-48ec-a320-103003694065', 'network_id': '77df64b5-a17d-48ec-a320-103003694065', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'fixed_ips': [{'subnet_id': '3d2ff612-067b-478d-bf96-a25c339bd70d'}]}} from dv-cinder-784092-9-iscsi. {{(pid=91821) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 11:08:12.626044 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:12.825583 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.db_base_plugin_common [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Allocated IP 172.20.0.2 (77df64b5-a17d-48ec-a320-103003694065/3d2ff612-067b-478d-bf96-a25c339bd70d/079b5c83-acce-4710-97ae-0210b66a7c97) {{(pid=91821) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:08:13.077608 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Notify callbacks [] for port, precommit_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:13.091011 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Transition to ACTIVE for port object 079b5c83-acce-4710-97ae-0210b66a7c97 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:08:13.110398 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Transition to ACTIVE for port object 079b5c83-acce-4710-97ae-0210b66a7c97 will not be triggered until provisioned by entity DHCP. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:08:13.117381 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:13.122155 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:13.184522 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Network 77df64b5-a17d-48ec-a320-103003694065 is already hosted by enough agents. {{(pid=91821) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:08:13.187325 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.base_scheduler [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Resources bound (agent type, host, resource id): {{(pid=91821) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:08:13.194442 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Received port after_create {{(pid=91821) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:08:13.196903 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Pushing event updated for resources: {'Port': ['ID=079b5c83-acce-4710-97ae-0210b66a7c97,revision_number=1']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:13.206636 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:13.206968 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:13.207303 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:13.207692 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:13.207941 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:13.208261 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:13.208685 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:13.208880 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:13.209220 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:13.209533 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:13.209856 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:13.210188 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Tracked resource versions report: May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: pushed versions: May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: { 'AddressGroup': {'1.2'}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'ConntrackHelper': {'1.0'}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Log': {'1.0'}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Network': {'1.1'}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Port': {'1.7'}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'PortForwarding': {'1.2'}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'QosPolicy': {'1.8'}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'SecurityGroup': {'1.4'}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'SecurityGroupRule': {'1.2'}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'SubPort': {'1.0'}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Subnet': {'1.1'}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Trunk': {'1.1'}} May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: consumer versions: May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: { DHCP agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: L3 agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: Metadata agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: Metering agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: Open vSwitch agent@dv-cinder-784092-9-iscsi: { 'AddressGroup': '1.2', May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'ConntrackHelper': '1.0', May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Log': '1.0', May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Network': '1.1', May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Port': '1.7', May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'PortForwarding': '1.2', May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'QosPolicy': '1.8', May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'SecurityGroup': '1.4', May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'SecurityGroupRule': '1.2', May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'SubPort': '1.0', May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Subnet': '1.1', May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Trunk': '1.1'}} May 24 11:08:13.212534 dv-cinder-784092-9-iscsi neutron-server[91821]: {{(pid=91821) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 11:08:13.246431 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.124s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:13.304297 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Attempting to bind port 079b5c83-acce-4710-97ae-0210b66a7c97 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:08:13.305018 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Attempting to bind port 079b5c83-acce-4710-97ae-0210b66a7c97 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '0e9fee1d-025f-4262-885a-f1e7a3dccf62', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 535, 'network_id': '77df64b5-a17d-48ec-a320-103003694065'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:08:13.305162 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Attempting to bind port 079b5c83-acce-4710-97ae-0210b66a7c97 on network 77df64b5-a17d-48ec-a320-103003694065 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:08:13.311368 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 7, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:08:13.311912 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Checking segment: {'id': '0e9fee1d-025f-4262-885a-f1e7a3dccf62', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 535, 'network_id': '77df64b5-a17d-48ec-a320-103003694065'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:08:13.312449 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:13.312847 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Bound using segment: {'id': '0e9fee1d-025f-4262-885a-f1e7a3dccf62', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 535, 'network_id': '77df64b5-a17d-48ec-a320-103003694065'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:08:13.313754 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Bound port: 079b5c83-acce-4710-97ae-0210b66a7c97, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '0e9fee1d-025f-4262-885a-f1e7a3dccf62', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 535, 'network_id': '77df64b5-a17d-48ec-a320-103003694065'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:08:13.313934 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:13.411604 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] For port 079b5c83-acce-4710-97ae-0210b66a7c97, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:08:13.415779 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] For port 079b5c83-acce-4710-97ae-0210b66a7c97, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=079b5c83-acce-4710-97ae-0210b66a7c97,segment=NetworkSegment(0e9fee1d-025f-4262-885a-f1e7a3dccf62),segment_id=0e9fee1d-025f-4262-885a-f1e7a3dccf62)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:08:13.420275 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:13.424444 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:13.531312 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Pushing event updated for resources: {'Port': ['ID=079b5c83-acce-4710-97ae-0210b66a7c97,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:13.534963 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.110s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:14.772353 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50028) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:15.159725 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [None req-581d49b0-6b34-44f6-8a95-c321654cb12d admin admin] http://10.222.0.18:9696/v2.0/extensions returned with HTTP 200 {{(pid=91820) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 11:08:15.160732 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-581d49b0-6b34-44f6-8a95-c321654cb12d admin admin] 10.222.0.18 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.3878527 May 24 11:08:17.605167 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50038) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:18.040337 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [None req-7f4696db-e8c4-4d44-8bb6-2bb41de47cff admin admin] http://10.222.0.18:9696/v2.0/extensions returned with HTTP 200 {{(pid=91820) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 11:08:18.041344 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-7f4696db-e8c4-4d44-8bb6-2bb41de47cff admin admin] 10.222.0.18 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.4352560 May 24 11:08:19.283628 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-719b1431-b649-4a2b-a6b2-258dc76abab1 None None] Network 77df64b5-a17d-48ec-a320-103003694065 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:08:19.366308 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-be9323ad-0764-44e0-9109-5bef8359bf64 None None] Provisioning for port 079b5c83-acce-4710-97ae-0210b66a7c97 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:08:20.075818 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 079b5c83-acce-4710-97ae-0210b66a7c97 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:08:20.144317 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 079b5c83-acce-4710-97ae-0210b66a7c97 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:08:20.149067 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 079b5c83-acce-4710-97ae-0210b66a7c97 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:08:20.149484 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:20.290221 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:20.323978 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 079b5c83-acce-4710-97ae-0210b66a7c97, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=079b5c83-acce-4710-97ae-0210b66a7c97,segment=NetworkSegment(0e9fee1d-025f-4262-885a-f1e7a3dccf62),segment_id=0e9fee1d-025f-4262-885a-f1e7a3dccf62)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:08:20.327829 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:20.330394 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:20.432916 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=079b5c83-acce-4710-97ae-0210b66a7c97,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:20.437805 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:20.766325 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50054) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:21.145524 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-73114059-f6d7-4794-a891-db68fb6e54e4 admin admin] Request body: {'router': {'name': 'router1', 'admin_state_up': True, 'tenant_id': '25622105db07401c9ad4153c1d775107'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:08:21.150380 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-73114059-f6d7-4794-a891-db68fb6e54e4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:08:21.154067 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-73114059-f6d7-4794-a891-db68fb6e54e4 admin admin] Usage tracker for resource:router and tenant:25622105db07401c9ad4153c1d775107 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:08:21.157327 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-73114059-f6d7-4794-a891-db68fb6e54e4 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:08:21.162844 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-73114059-f6d7-4794-a891-db68fb6e54e4 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:08:21.173251 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-73114059-f6d7-4794-a891-db68fb6e54e4 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-684314', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4157'] for router, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:21.174748 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-73114059-f6d7-4794-a891-db68fb6e54e4 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-684570', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-687045', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-715560', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4164'] for router, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:21.205501 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-73114059-f6d7-4794-a891-db68fb6e54e4 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:21.312471 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-73114059-f6d7-4794-a891-db68fb6e54e4 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-684307', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-687011'] for router, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:21.335143 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-73114059-f6d7-4794-a891-db68fb6e54e4 admin admin] Notify callbacks [] for router, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:21.336495 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-73114059-f6d7-4794-a891-db68fb6e54e4 admin admin] 10.222.0.18 "POST /v2.0/routers HTTP/1.1" status: 201 len: 709 time: 0.5688000 May 24 11:08:22.057540 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 079b5c83-acce-4710-97ae-0210b66a7c97 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:08:22.125844 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 079b5c83-acce-4710-97ae-0210b66a7c97 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:08:22.126219 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:23.373441 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-afa2045b-53c4-4cd1-8591-e234cfcfcb67 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:23.742908 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50064) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:24.177173 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [None req-07ef8b7d-4049-4354-85ee-6f4bea652f66 admin admin] http://10.222.0.18:9696/v2.0/extensions returned with HTTP 200 {{(pid=91820) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 11:08:24.178668 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-07ef8b7d-4049-4354-85ee-6f4bea652f66 admin admin] 10.222.0.18 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.4346817 May 24 11:08:25.749593 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:25.832257 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e7516949-8d81-46c0-a47b-389ea8d86873 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:26.619729 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 50074) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:26.891428 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:27.109620 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Request body: {'network': {'provider:network_type': 'flat', 'name': 'public', 'admin_state_up': True, 'is_default': True, 'provider:physical_network': 'physnet1', 'router:external': True}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:08:27.111103 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.quota [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Loaded quota_driver: . May 24 11:08:27.127177 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:08:27.133839 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Usage tracker for resource:network and tenant:6956503108cd48a5888e46f30578b371 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:08:27.139510 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:08:27.147457 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:08:27.163664 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Made reservation on behalf of 6956503108cd48a5888e46f30578b371 for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:08:27.237060 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.policy [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Loaded default policies from ['neutron'] under neutron.policies entry points {{(pid=91819) register_rules /opt/stack/neutron/neutron/policy.py:73}} May 24 11:08:27.283710 dv-cinder-784092-9-iscsi neutron-server[91819]: WARNING oslo_policy.policy [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 11:08:27.284084 dv-cinder-784092-9-iscsi neutron-server[91819]: WARNING oslo_policy.policy [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 24 11:08:27.294246 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:27.314111 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:27.329607 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:08:27.333473 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Usage tracker for resource:security_group_rule and tenant:6956503108cd48a5888e46f30578b371 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:08:27.336793 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:08:27.343225 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:08:27.450704 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:27.454692 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:27.460650 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:27.472350 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.type_flat [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Reserving flat network on physical network physnet1 {{(pid=91819) reserve_provider_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_flat.py:100}} May 24 11:08:27.507429 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Notify callbacks ['neutron.services.segments.db._add_segment_host_mapping_for_segment-8779719283694', 'neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:27.508106 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Unsubscribe: segment precommit_create {{(pid=91819) unsubscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:83}} May 24 11:08:27.508575 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Added segment eef8fa6c-c77b-4b44-a377-fa3b8b4035b0 of type flat for network 858b8fad-a73f-4578-9079-6e731b9ff7e2 May 24 11:08:27.564510 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=08a1a94e-8b92-4e93-8166-42f071ca6437,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:27.584057 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:27.584402 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:27.584728 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:27.585043 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:27.585379 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:27.585716 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:27.585995 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:27.586299 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:27.586601 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:27.586898 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:27.587198 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:27.587493 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Tracked resource versions report: May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: pushed versions: May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: { 'AddressGroup': {'1.2'}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'ConntrackHelper': {'1.0'}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Log': {'1.0'}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Network': {'1.1'}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Port': {'1.7'}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'PortForwarding': {'1.2'}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'QosPolicy': {'1.8'}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'SecurityGroup': {'1.4'}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'SecurityGroupRule': {'1.2'}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'SubPort': {'1.0'}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Subnet': {'1.1'}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Trunk': {'1.1'}} May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: consumer versions: May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: { DHCP agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: L3 agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: Metadata agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: Metering agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: Open vSwitch agent@dv-cinder-784092-9-iscsi: { 'AddressGroup': '1.2', May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'ConntrackHelper': '1.0', May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Log': '1.0', May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Network': '1.1', May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Port': '1.7', May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'PortForwarding': '1.2', May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'QosPolicy': '1.8', May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'SecurityGroup': '1.4', May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'SecurityGroupRule': '1.2', May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'SubPort': '1.0', May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Subnet': '1.1', May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Trunk': '1.1'}} May 24 11:08:27.588738 dv-cinder-784092-9-iscsi neutron-server[91819]: {{(pid=91819) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 11:08:27.645316 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:27.649362 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.189s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:27.692503 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:27.694405 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:08:27.695137 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:27.699970 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:27.810947 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 847 time: 1.1896656 May 24 11:08:27.811526 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Pushing event updated for resources: {'Network': ['ID=858b8fad-a73f-4578-9079-6e731b9ff7e2,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:27.814978 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.115s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:27.826491 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:08:28.799528 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:30.257384 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50100) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:30.475666 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a1f69175-1380-491d-9670-30b2d7273232 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:08:30.477016 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-a1f69175-1380-491d-9670-30b2d7273232 admin admin] 10.222.0.18 "GET /v2.0/subnets/3d2ff612-067b-478d-bf96-a25c339bd70d HTTP/1.1" status: 200 len: 849 time: 0.2187717 May 24 11:08:30.621348 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-8c0817eb-30af-44cb-8de3-be523808d35c admin admin] 10.222.0.18 "GET /v2.0/routers/c0b5ea83-a46a-4036-869b-ada12cd913b3 HTTP/1.1" status: 200 len: 704 time: 0.1384475 May 24 11:08:31.083326 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:31.344599 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Allocated IP 172.20.0.1 (77df64b5-a17d-48ec-a320-103003694065/3d2ff612-067b-478d-bf96-a25c339bd70d/c8733a53-8953-4b41-bde2-944fa1bba298) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:08:31.511804 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:31.529704 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Transition to ACTIVE for port object c8733a53-8953-4b41-bde2-944fa1bba298 will not be triggered until provisioned by entity DHCP. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:08:31.591078 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:31.595009 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:31.659886 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Network 77df64b5-a17d-48ec-a320-103003694065 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:08:31.662706 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:08:31.666401 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:08:31.666916 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-687570'] for router_interface, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:31.782246 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Pushing event updated for resources: {'Port': ['ID=c8733a53-8953-4b41-bde2-944fa1bba298,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:31.805202 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.210s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:31.908223 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-5a46bc00-adb7-4d47-b4c4-0b90fe79d681 None None] Provisioning for port c8733a53-8953-4b41-bde2-944fa1bba298 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:08:31.912945 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-5a46bc00-adb7-4d47-b4c4-0b90fe79d681 None None] Provisioning complete for port c8733a53-8953-4b41-bde2-944fa1bba298 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:08:31.913423 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-5a46bc00-adb7-4d47-b4c4-0b90fe79d681 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:31.927549 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:31.966625 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-5a46bc00-adb7-4d47-b4c4-0b90fe79d681 None None] Port c8733a53-8953-4b41-bde2-944fa1bba298 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:08:32.155453 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:32.159953 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:32.160973 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-533332', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-687487'] for router_interface, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:32.164370 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:32.297390 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Pushing event updated for resources: {'Port': ['ID=c8733a53-8953-4b41-bde2-944fa1bba298,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:32.301717 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.137s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:32.364131 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Network 77df64b5-a17d-48ec-a320-103003694065 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:08:32.365210 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:08:32.597625 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-6b4fea18-8e78-4c5d-87a2-08869e7367d1 None None] Provisioning complete for port c8733a53-8953-4b41-bde2-944fa1bba298 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:08:32.598038 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-6b4fea18-8e78-4c5d-87a2-08869e7367d1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:32.601286 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Router c0b5ea83-a46a-4036-869b-ada12cd913b3 is scheduled to L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 with binding_index 1 {{(pid=91820) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 11:08:32.655176 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-6b4fea18-8e78-4c5d-87a2-08869e7367d1 None None] Port c8733a53-8953-4b41-bde2-944fa1bba298 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:08:32.762549 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:08:32.771716 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] 10.222.0.18 "PUT /v2.0/routers/c0b5ea83-a46a-4036-869b-ada12cd913b3/add_router_interface HTTP/1.1" status: 200 len: 508 time: 2.1454375 May 24 11:08:32.774061 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Sync routers for ids ['c0b5ea83-a46a-4036-869b-ada12cd913b3'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:08:32.785493 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Routers {'c0b5ea83-a46a-4036-869b-ada12cd913b3'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:08:32.786272 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:08:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c0b5ea83-a46a-4036-869b-ada12cd913b3'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:08:32.973223 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Set router c0b5ea83-a46a-4036-869b-ada12cd913b3 gateway port host: None {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:08:33.350588 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'name': 'router1', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:08:21Z', 'updated_at': '2021-05-24T11:08:31Z', 'revision_number': 2, 'project_id': '25622105db07401c9ad4153c1d775107', 'gw_port_host': None, '_interfaces': [{'id': 'c8733a53-8953-4b41-bde2-944fa1bba298', 'name': '', 'network_id': '77df64b5-a17d-48ec-a320-103003694065', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'mac_address': 'fa:16:3e:ac:4e:ef', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '3d2ff612-067b-478d-bf96-a25c339bd70d', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:08:31Z', 'updated_at': '2021-05-24T11:08:31Z', 'revision_number': 2, 'project_id': '25622105db07401c9ad4153c1d775107', 'subnets': [{'id': '3d2ff612-067b-478d-bf96-a25c339bd70d', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'ce781e3a-a7da-4efc-9d7e-994a282506de'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:08:33.356165 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Checking router: c0b5ea83-a46a-4036-869b-ada12cd913b3 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:08:33.356532 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Updating router c0b5ea83-a46a-4036-869b-ada12cd913b3 port c8733a53-8953-4b41-bde2-944fa1bba298 binding host dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:08:33.413809 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:33.842347 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Notify callbacks [] for port, precommit_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:33.859385 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.provisioning_blocks [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Transition to ACTIVE for port object c8733a53-8953-4b41-bde2-944fa1bba298 will not be triggered until provisioned by entity L2. {{(pid=91822) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:08:33.867194 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:33.872878 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:34.091717 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Pushing event updated for resources: {'Port': ['ID=c8733a53-8953-4b41-bde2-944fa1bba298,revision_number=3']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:34.104648 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:34.104898 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:34.105253 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:34.105565 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:34.105875 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:34.106185 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:34.106494 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:34.106800 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:34.107113 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:34.107411 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:34.107714 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:34.108016 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Tracked resource versions report: May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: pushed versions: May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: { 'AddressGroup': {'1.2'}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'ConntrackHelper': {'1.0'}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Log': {'1.0'}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Network': {'1.1'}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Port': {'1.7'}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'PortForwarding': {'1.2'}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'QosPolicy': {'1.8'}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'SecurityGroup': {'1.4'}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'SecurityGroupRule': {'1.2'}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'SubPort': {'1.0'}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Subnet': {'1.1'}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Trunk': {'1.1'}} May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: consumer versions: May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: { DHCP agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: L3 agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: Metadata agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: Metering agent@dv-cinder-784092-9-iscsi: {}, May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: Open vSwitch agent@dv-cinder-784092-9-iscsi: { 'AddressGroup': '1.2', May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'ConntrackHelper': '1.0', May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Log': '1.0', May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Network': '1.1', May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Port': '1.7', May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'PortForwarding': '1.2', May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'QosPolicy': '1.8', May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'SecurityGroup': '1.4', May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'SecurityGroupRule': '1.2', May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'SubPort': '1.0', May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Subnet': '1.1', May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Trunk': '1.1'}} May 24 11:08:34.109112 dv-cinder-784092-9-iscsi neutron-server[91822]: {{(pid=91822) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 11:08:34.113273 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Attempting to bind port c8733a53-8953-4b41-bde2-944fa1bba298 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:08:34.113665 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Attempting to bind port c8733a53-8953-4b41-bde2-944fa1bba298 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '0e9fee1d-025f-4262-885a-f1e7a3dccf62', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 535, 'network_id': '77df64b5-a17d-48ec-a320-103003694065'}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:08:34.114123 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Attempting to bind port c8733a53-8953-4b41-bde2-944fa1bba298 on network 77df64b5-a17d-48ec-a320-103003694065 {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:08:34.116215 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.243s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:34.120792 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 8, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:08:34.121223 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Checking segment: {'id': '0e9fee1d-025f-4262-885a-f1e7a3dccf62', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 535, 'network_id': '77df64b5-a17d-48ec-a320-103003694065'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91822) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:08:34.121712 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:34.122098 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Bound using segment: {'id': '0e9fee1d-025f-4262-885a-f1e7a3dccf62', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 535, 'network_id': '77df64b5-a17d-48ec-a320-103003694065'} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:08:34.122624 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Bound port: c8733a53-8953-4b41-bde2-944fa1bba298, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '0e9fee1d-025f-4262-885a-f1e7a3dccf62', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 535, 'network_id': '77df64b5-a17d-48ec-a320-103003694065'}}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:08:34.122984 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:34.209903 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] For port c8733a53-8953-4b41-bde2-944fa1bba298, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91822) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:08:34.214589 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] For port c8733a53-8953-4b41-bde2-944fa1bba298, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=c8733a53-8953-4b41-bde2-944fa1bba298,segment=NetworkSegment(0e9fee1d-025f-4262-885a-f1e7a3dccf62),segment_id=0e9fee1d-025f-4262-885a-f1e7a3dccf62)] {{(pid=91822) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:08:34.219972 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:34.223947 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:34.324801 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Routers {'c0b5ea83-a46a-4036-869b-ada12cd913b3'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:08:34.327393 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:08:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c0b5ea83-a46a-4036-869b-ada12cd913b3'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:08:34.479821 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Pushing event updated for resources: {'Port': ['ID=c8733a53-8953-4b41-bde2-944fa1bba298,revision_number=4']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:34.488033 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.264s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:34.526531 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] Set router c0b5ea83-a46a-4036-869b-ada12cd913b3 gateway port host: None {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:08:34.851733 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'name': 'router1', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:08:21Z', 'updated_at': '2021-05-24T11:08:31Z', 'revision_number': 2, 'project_id': '25622105db07401c9ad4153c1d775107', 'gw_port_host': None, '_interfaces': [{'id': 'c8733a53-8953-4b41-bde2-944fa1bba298', 'name': '', 'network_id': '77df64b5-a17d-48ec-a320-103003694065', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'mac_address': 'fa:16:3e:ac:4e:ef', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '3d2ff612-067b-478d-bf96-a25c339bd70d', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:08:31Z', 'updated_at': '2021-05-24T11:08:34Z', 'revision_number': 4, 'project_id': '25622105db07401c9ad4153c1d775107', 'subnets': [{'id': '3d2ff612-067b-478d-bf96-a25c339bd70d', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'ce781e3a-a7da-4efc-9d7e-994a282506de'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:08:34.855683 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-44b8b2ca-043f-4e89-8e7f-02e5a24077f7 None None] The sync data for ids ['c0b5ea83-a46a-4036-869b-ada12cd913b3'] in dv-cinder-784092-9-iscsi is: [{'id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'name': 'router1', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:08:21Z', 'updated_at': '2021-05-24T11:08:31Z', 'revision_number': 2, 'project_id': '25622105db07401c9ad4153c1d775107', 'gw_port_host': None, '_interfaces': [{'id': 'c8733a53-8953-4b41-bde2-944fa1bba298', 'name': '', 'network_id': '77df64b5-a17d-48ec-a320-103003694065', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'mac_address': 'fa:16:3e:ac:4e:ef', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '3d2ff612-067b-478d-bf96-a25c339bd70d', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:08:31Z', 'updated_at': '2021-05-24T11:08:34Z', 'revision_number': 4, 'project_id': '25622105db07401c9ad4153c1d775107', 'subnets': [{'id': '3d2ff612-067b-478d-bf96-a25c339bd70d', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'ce781e3a-a7da-4efc-9d7e-994a282506de'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:08:34.993892 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 50124) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:35.147526 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-44fe9a8e-95b2-4490-b0e8-12e383fd838a admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:08:35.148036 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-44fe9a8e-95b2-4490-b0e8-12e383fd838a admin admin] Synchronizing usage tracker for tenant:6956503108cd48a5888e46f30578b371 on resource:network {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:08:35.167062 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-44fe9a8e-95b2-4490-b0e8-12e383fd838a admin admin] Unset dirty status for tenant:6956503108cd48a5888e46f30578b371 on resource:network {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:08:35.168781 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-44fe9a8e-95b2-4490-b0e8-12e383fd838a admin admin] 10.222.0.18 "GET /v2.0/networks/858b8fad-a73f-4578-9079-6e731b9ff7e2 HTTP/1.1" status: 200 len: 842 time: 0.1733441 May 24 11:08:35.178609 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Request body: {'subnet': {'enable_dhcp': False, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'gateway_ip': '172.27.0.1', 'cidr': '172.27.0.0/24', 'ip_version': 4, 'name': 'public-subnet'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:08:35.184262 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:08:35.188419 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Usage tracker for resource:subnet and tenant:6956503108cd48a5888e46f30578b371 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:08:35.191558 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:08:35.199411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:08:35.210237 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Made reservation on behalf of 6956503108cd48a5888e46f30578b371 for: {'subnet': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:08:35.212541 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:35.689654 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.objects.ports [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Time-cost: Fetching 0 ports in 0 security groups {{(pid=91819) get_objects /opt/stack/neutron/neutron/objects/ports.py:443}} May 24 11:08:35.690147 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781'] for subnet, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:35.695180 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:35.766736 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Pushing event updated for resources: {'Subnet': ['ID=a677ede7-728d-486b-819a-5fc603fb3f9b,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:35.770821 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Checking segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:08:35.771191 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Network hostable DHCP agents. Network: 858b8fad-a73f-4578-9079-6e731b9ff7e2, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91819) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:08:35.774886 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:35.796085 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91819) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:08:35.806748 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, 858b8fad-a73f-4578-9079-6e731b9ff7e2) {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:08:35.811456 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:08:35.812066 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:35.816880 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-b0127091-25dd-4e7f-bc1c-52b7699d5d5f None None] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:08:35.832736 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 821 time: 0.6580796 May 24 11:08:36.030389 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-f3cc1f31-2427-4e15-b4d4-69ec8cd64398 None None] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:08:38.079198 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device c8733a53-8953-4b41-bde2-944fa1bba298 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:08:38.121404 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 50138) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:38.149084 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port c8733a53-8953-4b41-bde2-944fa1bba298 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:08:38.153799 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port c8733a53-8953-4b41-bde2-944fa1bba298 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:08:38.154176 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:38.280864 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:38.320334 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port c8733a53-8953-4b41-bde2-944fa1bba298, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=c8733a53-8953-4b41-bde2-944fa1bba298,segment=NetworkSegment(0e9fee1d-025f-4262-885a-f1e7a3dccf62),segment_id=0e9fee1d-025f-4262-885a-f1e7a3dccf62)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:08:38.325544 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:38.327204 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:38.368608 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-2f7e1ba8-2acb-4b2f-9f01-46e0c9ef49e8 admin admin] 10.222.0.18 "GET /v2.0/routers/c0b5ea83-a46a-4036-869b-ada12cd913b3 HTTP/1.1" status: 200 len: 710 time: 0.2460346 May 24 11:08:38.396261 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=c8733a53-8953-4b41-bde2-944fa1bba298,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:38.399505 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:38.500632 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-3e951e25-8a8f-4f80-9a76-bb83845e7ef7 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:08:38.502237 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-3e951e25-8a8f-4f80-9a76-bb83845e7ef7 admin admin] 10.222.0.18 "GET /v2.0/networks/858b8fad-a73f-4578-9079-6e731b9ff7e2 HTTP/1.1" status: 200 len: 886 time: 0.1276724 May 24 11:08:38.510800 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Request body: {'router': {'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:08:38.966277 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-687072', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-684328', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4178'] for router, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:39.312756 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:39.372541 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:39.538597 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Allocated IP 172.27.0.222 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/cb66e5c7-ea37-480b-9209-57f7c9825be8) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:08:39.699035 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:39.755504 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:39.761950 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:39.825301 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:08:39.828007 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:08:39.831589 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:08:39.940169 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Pushing event updated for resources: {'Port': ['ID=cb66e5c7-ea37-480b-9209-57f7c9825be8,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:39.957824 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.199s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:40.075912 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device c8733a53-8953-4b41-bde2-944fa1bba298 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:08:40.146845 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port c8733a53-8953-4b41-bde2-944fa1bba298 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:08:40.147210 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:40.216089 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:40.559758 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-687072', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-684328', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4178'] for router, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:40.563638 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-687011', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_distributed_port_bindings_after_change-687083', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_snat_interfaces_after_change-687018', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._reconfigure_ha_resources-684350'] for router, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:40.746989 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Router c0b5ea83-a46a-4036-869b-ada12cd913b3 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:08:40.960011 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:08:40.968221 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Notify callbacks [] for router, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:40.969425 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] 10.222.0.18 "PUT /v2.0/routers/c0b5ea83-a46a-4036-869b-ada12cd913b3 HTTP/1.1" status: 200 len: 890 time: 2.4619401 May 24 11:08:40.970071 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Sync routers for ids ['c0b5ea83-a46a-4036-869b-ada12cd913b3'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:08:40.980771 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Routers {'c0b5ea83-a46a-4036-869b-ada12cd913b3'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:08:40.981546 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:08:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c0b5ea83-a46a-4036-869b-ada12cd913b3'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:08:41.203153 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Set router c0b5ea83-a46a-4036-869b-ada12cd913b3 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:08:41.563648 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'name': 'router1', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.222'}], 'enable_snat': True}, 'gw_port_id': 'cb66e5c7-ea37-480b-9209-57f7c9825be8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:08:21Z', 'updated_at': '2021-05-24T11:08:40Z', 'revision_number': 4, 'project_id': '25622105db07401c9ad4153c1d775107', 'gw_port': {'id': 'cb66e5c7-ea37-480b-9209-57f7c9825be8', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:ad:dd:d6', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.222', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:08:39Z', 'updated_at': '2021-05-24T11:08:39Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'c8733a53-8953-4b41-bde2-944fa1bba298', 'name': '', 'network_id': '77df64b5-a17d-48ec-a320-103003694065', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'mac_address': 'fa:16:3e:ac:4e:ef', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '3d2ff612-067b-478d-bf96-a25c339bd70d', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:08:31Z', 'updated_at': '2021-05-24T11:08:38Z', 'revision_number': 5, 'project_id': '25622105db07401c9ad4153c1d775107', 'subnets': [{'id': '3d2ff612-067b-478d-bf96-a25c339bd70d', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'ce781e3a-a7da-4efc-9d7e-994a282506de'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:08:41.568666 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Checking router: c0b5ea83-a46a-4036-869b-ada12cd913b3 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:08:41.569027 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Updating router c0b5ea83-a46a-4036-869b-ada12cd913b3 port cb66e5c7-ea37-480b-9209-57f7c9825be8 binding host dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:08:41.627688 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:41.965953 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Notify callbacks [] for port, precommit_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:41.979973 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Transition to ACTIVE for port object cb66e5c7-ea37-480b-9209-57f7c9825be8 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:08:41.987179 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:41.992149 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:42.068932 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Attempting to bind port cb66e5c7-ea37-480b-9209-57f7c9825be8 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:08:42.069337 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Attempting to bind port cb66e5c7-ea37-480b-9209-57f7c9825be8 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:08:42.069707 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Attempting to bind port cb66e5c7-ea37-480b-9209-57f7c9825be8 on network 858b8fad-a73f-4578-9079-6e731b9ff7e2 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:08:42.081244 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b3fa6816-0b68-48a8-920e-d32d0cfdb628 None None] Pushing event updated for resources: {'Port': ['ID=cb66e5c7-ea37-480b-9209-57f7c9825be8,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:42.083968 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b3fa6816-0b68-48a8-920e-d32d0cfdb628 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:42.084890 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 8, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:08:42.085299 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Checking segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:08:42.085779 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:42.086168 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Bound using segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:08:42.086641 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Bound port: cb66e5c7-ea37-480b-9209-57f7c9825be8, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:08:42.087020 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:42.166613 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] For port cb66e5c7-ea37-480b-9209-57f7c9825be8, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:08:42.169960 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] For port cb66e5c7-ea37-480b-9209-57f7c9825be8, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=cb66e5c7-ea37-480b-9209-57f7c9825be8,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:08:42.176922 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:42.183106 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:42.260988 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Routers {'c0b5ea83-a46a-4036-869b-ada12cd913b3'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:08:42.261663 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:08:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c0b5ea83-a46a-4036-869b-ada12cd913b3'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:08:42.309319 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b3fa6816-0b68-48a8-920e-d32d0cfdb628 None None] Pushing event updated for resources: {'Port': ['ID=cb66e5c7-ea37-480b-9209-57f7c9825be8,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:42.317378 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b3fa6816-0b68-48a8-920e-d32d0cfdb628 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.134s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:42.443059 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] Set router c0b5ea83-a46a-4036-869b-ada12cd913b3 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:08:42.808376 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'name': 'router1', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.222'}], 'enable_snat': True}, 'gw_port_id': 'cb66e5c7-ea37-480b-9209-57f7c9825be8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:08:21Z', 'updated_at': '2021-05-24T11:08:40Z', 'revision_number': 4, 'project_id': '25622105db07401c9ad4153c1d775107', 'gw_port': {'id': 'cb66e5c7-ea37-480b-9209-57f7c9825be8', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:ad:dd:d6', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.222', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:08:39Z', 'updated_at': '2021-05-24T11:08:42Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'c8733a53-8953-4b41-bde2-944fa1bba298', 'name': '', 'network_id': '77df64b5-a17d-48ec-a320-103003694065', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'mac_address': 'fa:16:3e:ac:4e:ef', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '3d2ff612-067b-478d-bf96-a25c339bd70d', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:08:31Z', 'updated_at': '2021-05-24T11:08:38Z', 'revision_number': 5, 'project_id': '25622105db07401c9ad4153c1d775107', 'subnets': [{'id': '3d2ff612-067b-478d-bf96-a25c339bd70d', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'ce781e3a-a7da-4efc-9d7e-994a282506de'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:08:42.812983 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-432c161f-1a4e-4157-85d4-71d87204ceba None None] The sync data for ids ['c0b5ea83-a46a-4036-869b-ada12cd913b3'] in dv-cinder-784092-9-iscsi is: [{'id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'name': 'router1', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.222'}], 'enable_snat': True}, 'gw_port_id': 'cb66e5c7-ea37-480b-9209-57f7c9825be8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:08:21Z', 'updated_at': '2021-05-24T11:08:40Z', 'revision_number': 4, 'project_id': '25622105db07401c9ad4153c1d775107', 'gw_port': {'id': 'cb66e5c7-ea37-480b-9209-57f7c9825be8', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:ad:dd:d6', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.222', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:08:39Z', 'updated_at': '2021-05-24T11:08:42Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'c8733a53-8953-4b41-bde2-944fa1bba298', 'name': '', 'network_id': '77df64b5-a17d-48ec-a320-103003694065', 'tenant_id': '25622105db07401c9ad4153c1d775107', 'mac_address': 'fa:16:3e:ac:4e:ef', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c0b5ea83-a46a-4036-869b-ada12cd913b3', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '3d2ff612-067b-478d-bf96-a25c339bd70d', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:08:31Z', 'updated_at': '2021-05-24T11:08:38Z', 'revision_number': 5, 'project_id': '25622105db07401c9ad4153c1d775107', 'subnets': [{'id': '3d2ff612-067b-478d-bf96-a25c339bd70d', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'ce781e3a-a7da-4efc-9d7e-994a282506de'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:08:43.350556 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50148) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:08:43.483566 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-9688c89b-f79b-491b-b764-5b4726946c24 admin admin] 10.222.0.18 "GET /v2.0/ports?device_owner=network%3Arouter_gateway&fields=id&fields=name&fields=mac_address&fields=fixed_ips&fields=status HTTP/1.1" status: 200 len: 411 time: 0.1317093 May 24 11:08:44.098215 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device cb66e5c7-ea37-480b-9209-57f7c9825be8 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:08:44.170641 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port cb66e5c7-ea37-480b-9209-57f7c9825be8 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:08:44.180466 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port cb66e5c7-ea37-480b-9209-57f7c9825be8 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:08:44.180851 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:44.307013 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:44.326731 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port cb66e5c7-ea37-480b-9209-57f7c9825be8, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=cb66e5c7-ea37-480b-9209-57f7c9825be8,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:08:44.331437 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:44.335987 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b3fa6816-0b68-48a8-920e-d32d0cfdb628 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:08:44.431469 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=cb66e5c7-ea37-480b-9209-57f7c9825be8,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:08:44.435307 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:08:46.089853 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device cb66e5c7-ea37-480b-9209-57f7c9825be8 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:08:46.178920 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port cb66e5c7-ea37-480b-9209-57f7c9825be8 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:08:46.179308 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:53.372488 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c30caada-48e2-40d6-974d-d20e6bb054e5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:55.282688 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 5 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:08:55.756052 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:55.827613 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-31869090-7c87-4d99-9ccf-065a311b4425 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:56.895645 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:08:58.827044 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:09:23.368624 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-324eacbc-d5fb-4608-90ad-60e0c2d7f86c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:09:25.753345 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:09:25.855377 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-280aee47-0e32-45e0-b8ac-33c8f8481ec1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:09:26.910391 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:09:28.836873 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:09:32.281508 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 5 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:09:53.373871 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-25696811-c9a9-4461-a877-920159babac3 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:09:55.750364 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:09:55.833157 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-85d922f3-02d2-4469-8bd1-3ecd973dae3e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:09:56.902566 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:09:58.828002 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:09.279977 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 5 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:10:22.385761 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50718) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:10:22.454528 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.pecan_wsgi.hooks.translation [None req-68c4ba89-7a7e-4ed4-960a-80b0677d8717 admin admin] GET failed (client error): The resource could not be found. May 24 11:10:22.455855 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-68c4ba89-7a7e-4ed4-960a-80b0677d8717 admin admin] 10.222.0.18 "GET /v2.0/networks/private HTTP/1.1" status: 404 len: 315 time: 0.0691528 May 24 11:10:22.598550 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-13ab659b-8b69-4ac4-ad49-ee75c37dbadf admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:22.599989 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-13ab659b-8b69-4ac4-ad49-ee75c37dbadf admin admin] 10.222.0.18 "GET /v2.0/networks?name=private HTTP/1.1" status: 200 len: 872 time: 0.1402166 May 24 11:10:23.379262 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17eacac7-7c6c-450d-aa25-0531e9b47a4b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:24.740532 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50728) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:10:24.852290 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-f4f16c73-9058-48bb-a17c-5239c6c65ccf admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:24.854072 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-f4f16c73-9058-48bb-a17c-5239c6c65ccf admin admin] 10.222.0.18 "GET /v2.0/subnets/3d2ff612-067b-478d-bf96-a25c339bd70d HTTP/1.1" status: 200 len: 849 time: 0.1125488 May 24 11:10:24.866288 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Request body: {'subnet': {'dns_nameservers': ['8.8.8.8', '8.8.4.4']}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:10:25.012044 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Notify callbacks [] for subnet, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:25.198062 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_subnet_gateway_ip_update-8779719271837'] for subnet, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:25.202533 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-9e5b3b44-65be-4bfe-ac14-94f660951e20 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:10:25.242456 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:25.243072 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:25.244324 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] 10.222.0.18 "PUT /v2.0/subnets/3d2ff612-067b-478d-bf96-a25c339bd70d HTTP/1.1" status: 200 len: 868 time: 0.3845739 May 24 11:10:25.247090 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-709e33a5-854e-465d-a2ad-9373b8ec63f6 None None] Network 77df64b5-a17d-48ec-a320-103003694065 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:10:25.269240 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Pushing event updated for resources: {'Subnet': ['ID=3d2ff612-067b-478d-bf96-a25c339bd70d,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:10:25.292151 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:10:25.292652 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:10:25.292991 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:10:25.293363 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:10:25.293686 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:10:25.294036 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:10:25.294355 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:10:25.294672 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:10:25.295003 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:10:25.295407 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:10:25.295722 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:10:25.296034 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:10:25.300142 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.097s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:10:25.748946 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:25.758209 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-e8092402-d83a-4722-884d-51b644c8fc4c None None] Provisioning complete for port 079b5c83-acce-4710-97ae-0210b66a7c97 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:10:25.759145 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e8092402-d83a-4722-884d-51b644c8fc4c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:25.841461 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-31c81fa4-9cf9-4408-b108-c41da624c7ac None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:25.912060 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-e8092402-d83a-4722-884d-51b644c8fc4c None None] Provisioning complete for port c8733a53-8953-4b41-bde2-944fa1bba298 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:10:25.912430 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e8092402-d83a-4722-884d-51b644c8fc4c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:26.910658 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:27.467442 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50738) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:10:27.653577 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0fe3cf63-8f28-440d-8bd0-9c60c7469b1f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:27.653983 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0fe3cf63-8f28-440d-8bd0-9c60c7469b1f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:27.655580 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-0fe3cf63-8f28-440d-8bd0-9c60c7469b1f admin admin] 10.222.0.18 "GET /v2.0/networks HTTP/1.1" status: 200 len: 1549 time: 0.1870694 May 24 11:10:28.835482 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:29.938946 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50748) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:10:30.129807 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-9804a989-e967-4893-928e-3b65a16f94bb admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:30.130285 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-9804a989-e967-4893-928e-3b65a16f94bb admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:30.131782 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-9804a989-e967-4893-928e-3b65a16f94bb admin admin] 10.222.0.18 "GET /v2.0/networks HTTP/1.1" status: 200 len: 1549 time: 0.1918061 May 24 11:10:32.345156 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 50758) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:10:32.525213 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-eada6eb6-01cf-4fa7-9e70-eed712deec75 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:32.526647 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-eada6eb6-01cf-4fa7-9e70-eed712deec75 admin admin] 10.222.0.18 "GET /v2.0/networks/77df64b5-a17d-48ec-a320-103003694065 HTTP/1.1" status: 200 len: 869 time: 0.1805282 May 24 11:10:34.680200 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 50768) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:10:34.847344 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-aee6514d-cbba-4938-b138-3265ba85cdf8 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:34.848643 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-aee6514d-cbba-4938-b138-3265ba85cdf8 admin admin] 10.222.0.18 "GET /v2.0/networks/858b8fad-a73f-4578-9079-6e731b9ff7e2 HTTP/1.1" status: 200 len: 886 time: 0.1667922 May 24 11:10:37.123105 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50778) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:10:37.235149 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a06926fe-73bb-4bef-b7fe-ad72211c3caf admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:37.235617 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a06926fe-73bb-4bef-b7fe-ad72211c3caf admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:37.237187 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-a06926fe-73bb-4bef-b7fe-ad72211c3caf admin admin] 10.222.0.18 "GET /v2.0/subnets HTTP/1.1" status: 200 len: 1479 time: 0.1131120 May 24 11:10:39.441240 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50788) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:10:39.548837 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d00279ce-4e66-4bb4-9c26-55a78987c536 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:39.549331 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d00279ce-4e66-4bb4-9c26-55a78987c536 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:39.550840 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-d00279ce-4e66-4bb4-9c26-55a78987c536 admin admin] 10.222.0.18 "GET /v2.0/subnets HTTP/1.1" status: 200 len: 1479 time: 0.1086802 May 24 11:10:41.710524 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50798) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:10:41.822813 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-45c6bf70-2e90-42f9-8b59-9338c2517e9b admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:41.824067 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-45c6bf70-2e90-42f9-8b59-9338c2517e9b admin admin] 10.222.0.18 "GET /v2.0/subnets/3d2ff612-067b-478d-bf96-a25c339bd70d HTTP/1.1" status: 200 len: 868 time: 0.1126080 May 24 11:10:43.964411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 50808) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:10:44.067495 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-5109cbf9-8eb6-436d-a17a-8eddd170b97e admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:10:44.068047 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-5109cbf9-8eb6-436d-a17a-8eddd170b97e admin admin] Synchronizing usage tracker for tenant:6956503108cd48a5888e46f30578b371 on resource:subnet {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:10:44.086550 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-5109cbf9-8eb6-436d-a17a-8eddd170b97e admin admin] Unset dirty status for tenant:6956503108cd48a5888e46f30578b371 on resource:subnet {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:10:44.087911 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-5109cbf9-8eb6-436d-a17a-8eddd170b97e admin admin] 10.222.0.18 "GET /v2.0/subnets/a677ede7-728d-486b-819a-5fc603fb3f9b HTTP/1.1" status: 200 len: 816 time: 0.1225595 May 24 11:10:46.279173 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 5 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:10:53.384318 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-14a4eecf-2a8c-4a72-a1fd-8994ad454223 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:55.746015 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:55.824985 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8bcc6780-95c0-42c2-b9fd-b4e6d3e6b926 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:56.917636 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:10:58.847768 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:08.734975 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 50852) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:11:08.815788 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [None req-4c462750-de1a-4dd3-90ad-9a709fe9120c admin admin] http://10.222.0.18:9696/v2.0/extensions returned with HTTP 200 {{(pid=91819) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 11:11:08.817778 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-4c462750-de1a-4dd3-90ad-9a709fe9120c admin admin] 10.222.0.18 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0808282 May 24 11:11:11.173688 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 50862) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:11:11.246661 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.pecan_wsgi.hooks.translation [None req-3534a9c5-13ee-47d4-8a67-d79081844224 admin admin] GET failed (client error): The resource could not be found. May 24 11:11:11.247991 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-3534a9c5-13ee-47d4-8a67-d79081844224 admin admin] 10.222.0.18 "GET /v2.0/networks/public HTTP/1.1" status: 404 len: 314 time: 0.0733719 May 24 11:11:11.383464 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b2803b2a-b455-4504-9703-c7f863ff2bb6 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:11:11.384432 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-b2803b2a-b455-4504-9703-c7f863ff2bb6 admin admin] 10.222.0.18 "GET /v2.0/networks?name=public HTTP/1.1" status: 200 len: 889 time: 0.1322141 May 24 11:11:13.595060 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 50872) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:11:13.645787 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Request body: {'network': {'name': 'shared', 'shared': True, 'admin_state_up': True}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:11:13.650883 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:11:13.660683 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Attempting to reserve 1 items for resource network. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:11:13.671511 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Made reservation on behalf of 6956503108cd48a5888e46f30578b371 for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:11:13.674190 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:13.763565 dv-cinder-784092-9-iscsi neutron-server[91819]: WARNING neutron.services.revisions.revision_plugin [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Could not find related network for resource to bump revision. May 24 11:11:13.777554 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] vlan segment allocate from pool success with {'vlan_id': 504, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:11:13.788417 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:13.788797 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Added segment 18940746-bb2a-4fb9-aa85-f52be81dada5 of type vlan for network f5351ce6-b1a3-4ac8-a776-f21bf320429a May 24 11:11:13.818631 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:13.850149 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:13.851449 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:11:13.852527 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:13.855715 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:11:13.963462 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 847 time: 0.3672447 May 24 11:11:13.966313 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Pushing event updated for resources: {'Network': ['ID=f5351ce6-b1a3-4ac8-a776-f21bf320429a,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:11:13.973711 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:13.974065 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:13.974386 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:13.974761 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:13.975087 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:13.975409 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:13.975725 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:13.976042 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:13.976434 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:13.976793 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:13.977663 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:13.977974 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:13.985583 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:11:13.996528 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network f5351ce6-b1a3-4ac8-a776-f21bf320429a details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:11:16.195740 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 50882) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:11:16.268236 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.pecan_wsgi.hooks.translation [None req-335f2336-429c-4a1a-a724-580c5f3d85b3 admin admin] GET failed (client error): The resource could not be found. May 24 11:11:16.268782 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-335f2336-429c-4a1a-a724-580c5f3d85b3 admin admin] Synchronizing usage tracker for tenant:6956503108cd48a5888e46f30578b371 on resource:network {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:11:16.285317 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-335f2336-429c-4a1a-a724-580c5f3d85b3 admin admin] Unset dirty status for tenant:6956503108cd48a5888e46f30578b371 on resource:network {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:11:16.288143 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-335f2336-429c-4a1a-a724-580c5f3d85b3 admin admin] 10.222.0.18 "GET /v2.0/networks/shared HTTP/1.1" status: 404 len: 314 time: 0.0898149 May 24 11:11:16.399592 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a496f8f2-39ad-4541-a396-47ae47c296fa admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:11:16.401074 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-a496f8f2-39ad-4541-a396-47ae47c296fa admin admin] 10.222.0.18 "GET /v2.0/networks?name=shared HTTP/1.1" status: 200 len: 826 time: 0.1098924 May 24 11:11:16.411385 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Request body: {'subnet': {'ip_version': 4, 'description': 'shared-subnet', 'name': 'shared-subnet', 'cidr': '192.168.233.0/24', 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:11:16.416796 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:11:16.426865 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:11:16.436747 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Made reservation on behalf of 6956503108cd48a5888e46f30578b371 for: {'subnet': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:11:16.439111 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Notify callbacks [] for subnet, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:16.883588 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781'] for subnet, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:16.887465 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-11bc5fe2-6f9a-49d0-9449-a3609bb43498 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:11:16.963272 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Pushing event updated for resources: {'Subnet': ['ID=6328b963-5efa-47e7-8def-4502c360c671,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:11:16.966717 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:11:16.973177 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Checking segment: {'id': '18940746-bb2a-4fb9-aa85-f52be81dada5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 504, 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:11:16.973539 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Network hostable DHCP agents. Network: f5351ce6-b1a3-4ac8-a776-f21bf320429a, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91819) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:11:16.991852 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Network f5351ce6-b1a3-4ac8-a776-f21bf320429a is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91819) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:11:17.014218 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, f5351ce6-b1a3-4ac8-a776-f21bf320429a) {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:11:17.018826 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:11:17.019703 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:17.026097 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-21df3387-9667-4a76-b7a0-683fdfacffbb None None] Network f5351ce6-b1a3-4ac8-a776-f21bf320429a requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:11:17.049616 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 0.6417718 May 24 11:11:17.309045 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpea482e27-f15a-5756-a111-7e0dba035002-f5351ce6-b1a3-4ac8-a776-f21bf320429a', 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a', 'tenant_id': '6956503108cd48a5888e46f30578b371', 'fixed_ips': [{'subnet_id': '6328b963-5efa-47e7-8def-4502c360c671'}]}} from dv-cinder-784092-9-iscsi. {{(pid=91821) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 11:11:17.317829 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:17.498807 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.db_base_plugin_common [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Allocated IP 192.168.233.2 (f5351ce6-b1a3-4ac8-a776-f21bf320429a/6328b963-5efa-47e7-8def-4502c360c671/ef3fba05-3a87-4edc-8d4a-ffeb6e482e63) {{(pid=91821) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:11:17.745814 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Notify callbacks [] for port, precommit_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:17.758242 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Transition to ACTIVE for port object ef3fba05-3a87-4edc-8d4a-ffeb6e482e63 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:11:17.786759 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Transition to ACTIVE for port object ef3fba05-3a87-4edc-8d4a-ffeb6e482e63 will not be triggered until provisioned by entity DHCP. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:11:17.793840 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:17.799975 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:11:17.870340 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Network f5351ce6-b1a3-4ac8-a776-f21bf320429a is already hosted by enough agents. {{(pid=91821) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:11:17.873287 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.base_scheduler [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Resources bound (agent type, host, resource id): {{(pid=91821) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:11:17.879188 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Received port after_create {{(pid=91821) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:11:17.886004 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Pushing event updated for resources: {'Port': ['ID=ef3fba05-3a87-4edc-8d4a-ffeb6e482e63,revision_number=1']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:11:17.925928 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:17.926275 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:17.926613 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:17.926942 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:17.927262 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:17.927587 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:17.927907 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:17.928229 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:17.928567 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:17.928881 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:17.929219 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:17.929533 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:11:17.937432 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.137s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:11:17.989780 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Attempting to bind port ef3fba05-3a87-4edc-8d4a-ffeb6e482e63 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:11:17.990148 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Attempting to bind port ef3fba05-3a87-4edc-8d4a-ffeb6e482e63 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '18940746-bb2a-4fb9-aa85-f52be81dada5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 504, 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:11:17.990510 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Attempting to bind port ef3fba05-3a87-4edc-8d4a-ffeb6e482e63 on network f5351ce6-b1a3-4ac8-a776-f21bf320429a {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:11:17.996629 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 10, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:11:17.997010 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Checking segment: {'id': '18940746-bb2a-4fb9-aa85-f52be81dada5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 504, 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:11:17.997454 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:17.997870 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Bound using segment: {'id': '18940746-bb2a-4fb9-aa85-f52be81dada5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 504, 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:11:17.998348 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Bound port: ef3fba05-3a87-4edc-8d4a-ffeb6e482e63, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '18940746-bb2a-4fb9-aa85-f52be81dada5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 504, 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:11:17.998727 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:18.097152 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] For port ef3fba05-3a87-4edc-8d4a-ffeb6e482e63, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:11:18.100723 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] For port ef3fba05-3a87-4edc-8d4a-ffeb6e482e63, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=ef3fba05-3a87-4edc-8d4a-ffeb6e482e63,segment=NetworkSegment(18940746-bb2a-4fb9-aa85-f52be81dada5),segment_id=18940746-bb2a-4fb9-aa85-f52be81dada5)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:11:18.104612 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:18.108797 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:11:18.192009 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Pushing event updated for resources: {'Port': ['ID=ef3fba05-3a87-4edc-8d4a-ffeb6e482e63,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:11:18.192244 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:11:19.045668 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-3f56bff6-5be0-44bc-ba73-76e2c1cc6057 None None] Network f5351ce6-b1a3-4ac8-a776-f21bf320429a requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:11:19.141279 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-4db2cf70-28f4-41e7-b881-fe4f035b4396 None None] Provisioning for port ef3fba05-3a87-4edc-8d4a-ffeb6e482e63 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:11:20.228891 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device ef3fba05-3a87-4edc-8d4a-ffeb6e482e63 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:11:20.313217 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port ef3fba05-3a87-4edc-8d4a-ffeb6e482e63 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:11:20.319884 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port ef3fba05-3a87-4edc-8d4a-ffeb6e482e63 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:11:20.320284 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:20.468114 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:20.506185 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port ef3fba05-3a87-4edc-8d4a-ffeb6e482e63, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=ef3fba05-3a87-4edc-8d4a-ffeb6e482e63,segment=NetworkSegment(18940746-bb2a-4fb9-aa85-f52be81dada5),segment_id=18940746-bb2a-4fb9-aa85-f52be81dada5)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:11:20.512010 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:20.515765 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:11:20.587358 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=ef3fba05-3a87-4edc-8d4a-ffeb6e482e63,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:11:20.590787 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:11:22.187239 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device ef3fba05-3a87-4edc-8d4a-ffeb6e482e63 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:11:22.284184 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port ef3fba05-3a87-4edc-8d4a-ffeb6e482e63 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:11:22.284556 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:23.279773 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 5 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:11:23.388695 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-ef4dcac9-2bec-4eaf-b60c-91e424e9dbc7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:25.741557 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:25.830615 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-88e78ec4-246d-4aec-bfd8-f7f7ccf16d4c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:26.912644 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:28.845586 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:53.410118 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d722eb61-b85e-4928-847c-cee4a464c549 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:55.734695 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:55.842419 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c6e6db56-21f2-4e37-bac1-14a3e7e3afa2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:56.916526 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:11:58.862225 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:00.281214 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 5 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:12:23.399796 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f2655e0a-fa4b-40f5-a10d-508ae5cfb118 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:25.731993 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:25.838379 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-67b1d246-04e8-4e91-a523-a0dad07cecff None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:26.917128 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:28.857042 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:32.721273 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 50932) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:12:32.731188 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [None req-7b2900ab-ecc3-4c6f-a6b8-67cfff8069ef tempest-verify_tempest_config-2141344905 tempest-verify_tempest_config-2141344905-project] http://10.222.0.18:9696/v2.0/extensions returned with HTTP 200 {{(pid=91820) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 11:12:32.732808 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-7b2900ab-ecc3-4c6f-a6b8-67cfff8069ef tempest-verify_tempest_config-2141344905 tempest-verify_tempest_config-2141344905-project] 10.222.0.18 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13635 time: 0.0103016 May 24 11:12:33.172416 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 50950) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:12:33.293411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:33.308997 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:12:33.312983 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Usage tracker for resource:security_group_rule and tenant:4e2857c870484e4682654eae6e267864 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:12:33.316412 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:12:33.322314 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:12:33.407798 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:33.411882 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:33.420378 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:12:33.459267 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Synchronizing usage tracker for tenant:6956503108cd48a5888e46f30578b371 on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:12:33.463134 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0781467c-df2c-464c-bb02-f8db967085e7,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:12:33.474900 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:12:33.475263 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:12:33.475599 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:12:33.476551 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:12:33.476878 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:12:33.477246 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:12:33.477569 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:12:33.477884 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:12:33.478216 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:12:33.478532 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:12:33.478848 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:12:33.479156 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:12:33.484893 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.065s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:12:33.491566 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Unset dirty status for tenant:6956503108cd48a5888e46f30578b371 on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:12:33.492876 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=4e2857c870484e4682654eae6e267864&name=default HTTP/1.1" status: 200 len: 2975 time: 0.3194332 May 24 11:12:33.495940 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 50956) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:12:33.557983 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b71326ea-6b03-45cf-b5bf-1ea7f8380555 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:33.583871 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b71326ea-6b03-45cf-b5bf-1ea7f8380555 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:33.680576 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b71326ea-6b03-45cf-b5bf-1ea7f8380555 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:33.684408 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9d65b35c-7471-4f7a-bdd4-3243a92dbaf7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:12:33.697911 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b71326ea-6b03-45cf-b5bf-1ea7f8380555 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0781467c-df2c-464c-bb02-f8db967085e7,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:12:33.701260 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b71326ea-6b03-45cf-b5bf-1ea7f8380555 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:12:33.703261 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b71326ea-6b03-45cf-b5bf-1ea7f8380555 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:33.704362 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-b71326ea-6b03-45cf-b5bf-1ea7f8380555 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/0781467c-df2c-464c-bb02-f8db967085e7 HTTP/1.1" status: 204 len: 168 time: 0.2074869 May 24 11:12:37.282967 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 5 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:12:53.391931 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-28a806a2-3ba9-4fda-b4e3-e43d2706622e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:55.736071 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:55.841707 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-59cc9e9c-ffa2-4eae-9e69-92205ff7d1d2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:56.928978 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:12:58.860190 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:13:14.282184 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 5 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:13:23.394527 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-2417a6e2-4423-4205-bd20-d7d8c24af1c9 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:13:25.743078 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:13:25.852772 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-fcf89f54-4809-4727-a1b3-a63c80a5da61 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:13:26.922187 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:13:28.864478 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:13:51.283113 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 5 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:13:53.398314 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1f4f13f0-e433-43f2-a295-7314bc28478a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:13:55.737074 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:13:55.845877 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f83625bc-4818-4010-8218-75b64e5a5bc2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:13:56.927146 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:13:58.864298 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:14:21.391030 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703'] for agent, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:14:23.400578 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-fab52c4a-9057-4b64-a5cd-2827641a46be None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:14:25.741373 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:14:25.847928 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8dc07915-f5ee-43e9-9806-26b7baff46f4 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:14:25.916742 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51008) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:14:26.038176 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-fd41311c-5772-4f9c-94aa-c2d82c78d381 admin admin] 10.222.0.18 "GET /v2.0/agents HTTP/1.1" status: 200 len: 4303 time: 0.1202533 May 24 11:14:26.927761 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:14:28.267525 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51018) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:14:28.283082 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:14:28.327062 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-1e21e8da-c1ce-4cba-bcfa-60191ee36d25 admin admin] 10.222.0.18 "GET /v2.0/agents HTTP/1.1" status: 200 len: 4303 time: 0.0583944 May 24 11:14:28.863411 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:14:51.360289 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:14:53.403757 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-48fbcb26-3215-4883-a53f-195a8728c3f7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:14:55.748259 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:14:55.851389 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-6fbea3df-af37-459f-a903-c95982514d2d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:14:56.933116 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:14:58.867588 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:05.286655 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:15:21.376548 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:23.410369 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d1b65514-c3a4-4c9b-8b1f-bc71367a0a6d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:25.743563 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:25.854063 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1d12886d-6aea-4abe-b8d1-9d0566daab6f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:26.933012 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:28.871984 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:42.287958 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:15:51.373227 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:53.414427 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-ca54ed97-8f64-4dfa-9275-99a7100098df None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:55.763896 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:55.869272 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a92bc1a3-394b-43e3-b2fd-008314be212c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:56.238141 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51096) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:15:56.406413 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeTestJSON-1010114930-network', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:15:56.421644 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:15:56.435599 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Usage tracker for resource:network and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:15:56.439385 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:15:56.446886 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:15:56.461669 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Made reservation on behalf of b303f83a84794e5ba6a1bab9c76b429c for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:15:56.467415 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:56.490194 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:56.513926 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:15:56.519092 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Usage tracker for resource:security_group_rule and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:15:56.523464 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:15:56.532329 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:15:56.674241 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:56.682475 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:56.689998 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:15:56.717640 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] vlan segment allocate from pool success with {'vlan_id': 518, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:15:56.748389 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f9e46c40-bf1d-4484-bb18-341b351d3f29,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:15:56.751747 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:56.753564 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Added segment bbef7673-75cf-415f-81a6-941c270eac8b of type vlan for network 1d92641a-d8f9-4ed6-9abd-f464bb26811d May 24 11:15:56.767050 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:56.767050 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:56.767050 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:56.767050 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:56.767050 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:56.767050 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:56.767050 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:56.768015 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:56.768345 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:56.768869 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:56.769264 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:56.769854 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Tracked resource versions report: May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: pushed versions: May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: { 'AddressGroup': {'1.2'}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'ConntrackHelper': {'1.0'}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Log': {'1.0'}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Network': {'1.1'}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Port': {'1.7'}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'PortForwarding': {'1.2'}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'QosPolicy': {'1.8'}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'SecurityGroup': {'1.4'}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'SecurityGroupRule': {'1.2'}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'SubPort': {'1.0'}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Subnet': {'1.1'}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Trunk': {'1.1'}} May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: consumer versions: May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: { DHCP agent@dv-cinder-784092-9-iscsi: {}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: HyperV agent@hv-cinder-78409: {}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: L3 agent@dv-cinder-784092-9-iscsi: {}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: Metadata agent@dv-cinder-784092-9-iscsi: {}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: Metering agent@dv-cinder-784092-9-iscsi: {}, May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: Open vSwitch agent@dv-cinder-784092-9-iscsi: { 'AddressGroup': '1.2', May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'ConntrackHelper': '1.0', May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Log': '1.0', May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Network': '1.1', May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Port': '1.7', May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'PortForwarding': '1.2', May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'QosPolicy': '1.8', May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'SecurityGroup': '1.4', May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'SecurityGroupRule': '1.2', May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'SubPort': '1.0', May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Subnet': '1.1', May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: 'Trunk': '1.1'}} May 24 11:15:56.771460 dv-cinder-784092-9-iscsi neutron-server[91819]: {{(pid=91819) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 11:15:56.779280 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:15:56.813940 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:56.859436 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:56.861299 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:15:56.862009 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:56.866268 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:15:56.948764 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:56.988032 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51148) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:15:56.990140 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 884 time: 0.7502439 May 24 11:15:56.994253 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Request body: {'subnet': {'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeTestJSON-1010114930-subnet', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'enable_dhcp': True, 'ip_version': 4}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:15:56.997424 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Pushing event updated for resources: {'Network': ['ID=1d92641a-d8f9-4ed6-9abd-f464bb26811d,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:15:56.999553 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:15:57.000967 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.135s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:15:57.013784 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network 1d92641a-d8f9-4ed6-9abd-f464bb26811d details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:15:57.017405 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Usage tracker for resource:subnet and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:15:57.020785 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:15:57.027794 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:15:57.040093 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Made reservation on behalf of b303f83a84794e5ba6a1bab9c76b429c for: {'subnet': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:15:57.042936 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:57.567931 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781'] for subnet, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:57.573219 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:15:57.630303 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Pushing event updated for resources: {'Subnet': ['ID=2fc4be2e-875c-4d10-b857-a48f90c53cf2,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:15:57.643275 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:57.643637 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:57.643969 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:57.644290 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:57.644588 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:57.644926 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:57.645267 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:57.645600 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:57.645927 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:57.646238 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:57.646692 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:57.646999 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Tracked resource versions report: May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: pushed versions: May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: { 'AddressGroup': {'1.2'}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'ConntrackHelper': {'1.0'}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Log': {'1.0'}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Network': {'1.1'}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Port': {'1.7'}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'PortForwarding': {'1.2'}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'QosPolicy': {'1.8'}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'SecurityGroup': {'1.4'}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'SecurityGroupRule': {'1.2'}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'SubPort': {'1.0'}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Subnet': {'1.1'}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Trunk': {'1.1'}} May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: consumer versions: May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: { DHCP agent@dv-cinder-784092-9-iscsi: {}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: HyperV agent@hv-cinder-78409: {}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: L3 agent@dv-cinder-784092-9-iscsi: {}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: Metadata agent@dv-cinder-784092-9-iscsi: {}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: Metering agent@dv-cinder-784092-9-iscsi: {}, May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: Open vSwitch agent@dv-cinder-784092-9-iscsi: { 'AddressGroup': '1.2', May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'ConntrackHelper': '1.0', May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Log': '1.0', May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Network': '1.1', May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Port': '1.7', May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'PortForwarding': '1.2', May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'QosPolicy': '1.8', May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'SecurityGroup': '1.4', May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'SecurityGroupRule': '1.2', May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'SubPort': '1.0', May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Subnet': '1.1', May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: 'Trunk': '1.1'}} May 24 11:15:57.648012 dv-cinder-784092-9-iscsi neutron-server[91820]: {{(pid=91820) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 11:15:57.660497 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:15:57.673580 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Checking segment: {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:15:57.673920 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Network hostable DHCP agents. Network: 1d92641a-d8f9-4ed6-9abd-f464bb26811d, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91820) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:15:57.692519 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Network 1d92641a-d8f9-4ed6-9abd-f464bb26811d is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91820) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:15:57.703983 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, 1d92641a-d8f9-4ed6-9abd-f464bb26811d) {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:15:57.708334 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:15:57.709095 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:57.726221 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-53d0acc5-e92a-4820-9249-dd6ef737f2eb None None] Network 1d92641a-d8f9-4ed6-9abd-f464bb26811d requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:15:57.732250 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 847 time: 0.7430940 May 24 11:15:57.735810 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51180) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:15:57.745858 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Request body: {'router': {'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:15:57.751806 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:15:57.757837 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Usage tracker for resource:router and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:15:57.762667 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:15:57.770190 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:15:57.789348 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-684314', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4157'] for router, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:57.790469 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-684570', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-687045', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-715560', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4164'] for router, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:57.822007 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:57.985252 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpea482e27-f15a-5756-a111-7e0dba035002-1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2'}]}} from dv-cinder-784092-9-iscsi. {{(pid=91821) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 11:15:57.986940 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:58.184936 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-687072', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-684328', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4178'] for router, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:58.346031 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.db_base_plugin_common [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Allocated IP 172.20.0.2 (1d92641a-d8f9-4ed6-9abd-f464bb26811d/2fc4be2e-875c-4d10-b857-a48f90c53cf2/0377cd0a-2401-44a9-a8d2-cc3ce593483b) {{(pid=91821) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:15:58.588830 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:58.634442 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:58.657755 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Notify callbacks [] for port, precommit_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:58.673182 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Transition to ACTIVE for port object 0377cd0a-2401-44a9-a8d2-cc3ce593483b will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:15:58.699230 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Transition to ACTIVE for port object 0377cd0a-2401-44a9-a8d2-cc3ce593483b will not be triggered until provisioned by entity DHCP. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:15:58.706807 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:58.711996 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:15:58.782114 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Network 1d92641a-d8f9-4ed6-9abd-f464bb26811d is already hosted by enough agents. {{(pid=91821) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:15:58.785395 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.base_scheduler [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Resources bound (agent type, host, resource id): {{(pid=91821) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:15:58.790459 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Received port after_create {{(pid=91821) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:15:58.800049 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Pushing event updated for resources: {'Port': ['ID=0377cd0a-2401-44a9-a8d2-cc3ce593483b,revision_number=1']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:15:58.841804 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:58.842175 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:58.842543 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:58.842869 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:58.843194 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:58.843547 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:58.843979 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:58.844304 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:58.844626 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:58.844943 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:58.845438 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:58.845861 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Tracked resource versions report: May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: pushed versions: May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: { 'AddressGroup': {'1.2'}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'ConntrackHelper': {'1.0'}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Log': {'1.0'}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Network': {'1.1'}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Port': {'1.7'}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'PortForwarding': {'1.2'}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'QosPolicy': {'1.8'}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'SecurityGroup': {'1.4'}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'SecurityGroupRule': {'1.2'}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'SubPort': {'1.0'}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Subnet': {'1.1'}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Trunk': {'1.1'}} May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: consumer versions: May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: { DHCP agent@dv-cinder-784092-9-iscsi: {}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: HyperV agent@hv-cinder-78409: {}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: L3 agent@dv-cinder-784092-9-iscsi: {}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: Metadata agent@dv-cinder-784092-9-iscsi: {}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: Metering agent@dv-cinder-784092-9-iscsi: {}, May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: Open vSwitch agent@dv-cinder-784092-9-iscsi: { 'AddressGroup': '1.2', May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'ConntrackHelper': '1.0', May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Log': '1.0', May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Network': '1.1', May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Port': '1.7', May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'PortForwarding': '1.2', May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'QosPolicy': '1.8', May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'SecurityGroup': '1.4', May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'SecurityGroupRule': '1.2', May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'SubPort': '1.0', May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Subnet': '1.1', May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: 'Trunk': '1.1'}} May 24 11:15:58.848003 dv-cinder-784092-9-iscsi neutron-server[91821]: {{(pid=91821) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 11:15:58.870443 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51228) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:15:58.898298 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.187s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:15:58.906271 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Allocated IP 172.27.0.39 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/541360c9-23fa-4c8b-9a48-116e140aa6a4) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:15:58.930432 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Attempting to bind port 0377cd0a-2401-44a9-a8d2-cc3ce593483b on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:15:58.930804 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Attempting to bind port 0377cd0a-2401-44a9-a8d2-cc3ce593483b by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:15:58.931521 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Attempting to bind port 0377cd0a-2401-44a9-a8d2-cc3ce593483b on network 1d92641a-d8f9-4ed6-9abd-f464bb26811d {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:15:58.934525 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:58.939356 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 15, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:15:58.939729 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Checking segment: {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:15:58.940130 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:58.940528 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Bound using segment: {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:15:58.941004 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Bound port: 0377cd0a-2401-44a9-a8d2-cc3ce593483b, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:15:58.941803 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.035195 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] For port 0377cd0a-2401-44a9-a8d2-cc3ce593483b, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:15:59.039473 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] For port 0377cd0a-2401-44a9-a8d2-cc3ce593483b, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=0377cd0a-2401-44a9-a8d2-cc3ce593483b,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:15:59.044811 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.055679 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:15:59.109017 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.135555 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51240) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:15:59.150200 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.158965 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-fa84bdf0-dbe8-428d-92fb-f1e4d98f56ca admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:15:59.195150 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Pushing event updated for resources: {'Port': ['ID=0377cd0a-2401-44a9-a8d2-cc3ce593483b,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:15:59.198912 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.143s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:15:59.242006 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:15:59.245185 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:15:59.249244 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:15:59.338865 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51250) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:15:59.373541 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Pushing event updated for resources: {'Port': ['ID=541360c9-23fa-4c8b-9a48-116e140aa6a4,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:15:59.417135 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51256) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:15:59.467725 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.309s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:15:59.472737 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Request body: {'network': {'name': 'tempest-EncryptedVolumesExtendAttachedTest-1271840068-network', 'tenant_id': 'de2d8ae871924896af66ae6c97b6ee1b'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:15:59.480647 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:15:59.514421 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Request body: {'network': {'name': 'tempest-ServerRescueNegativeTestJSON-12783889-network', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:15:59.521773 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:15:59.527592 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Usage tracker for resource:network and tenant:bf5fa11a013d4e1a8b86edc5fe49e97b is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:15:59.529687 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Usage tracker for resource:network and tenant:de2d8ae871924896af66ae6c97b6ee1b is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:15:59.536130 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:15:59.538541 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:15:59.549656 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:15:59.553495 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:15:59.581389 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Made reservation on behalf of bf5fa11a013d4e1a8b86edc5fe49e97b for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:15:59.586827 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Made reservation on behalf of de2d8ae871924896af66ae6c97b6ee1b for: {'network': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:15:59.587783 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.589974 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.609042 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.611611 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.635356 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:15:59.642352 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Usage tracker for resource:security_group_rule and tenant:bf5fa11a013d4e1a8b86edc5fe49e97b is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:15:59.648006 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:15:59.661052 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:15:59.763840 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:15:59.787579 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Usage tracker for resource:security_group_rule and tenant:de2d8ae871924896af66ae6c97b6ee1b is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:15:59.790800 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.794124 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:15:59.798677 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.804968 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b71326ea-6b03-45cf-b5bf-1ea7f8380555 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:15:59.807265 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:15:59.848204 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Request body: {'network': {'name': 'tempest-DeleteServersTestJSON-1595921365-network', 'tenant_id': 'e65fade283614b028730cfea0f58bcad'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:15:59.852444 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] vlan segment allocate from pool success with {'vlan_id': 519, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:15:59.863702 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=cbdda050-525d-4636-8079-44cfb0e04b03,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:15:59.870490 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:15:59.872923 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.873590 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Added segment 6bc6a763-352e-4ddf-bea5-3721ee687b85 of type vlan for network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 May 24 11:15:59.875297 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:15:59.887293 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Usage tracker for resource:network and tenant:e65fade283614b028730cfea0f58bcad is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:15:59.890177 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.892907 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:15:59.905757 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:15:59.926552 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.940272 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Made reservation on behalf of e65fade283614b028730cfea0f58bcad for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:15:59.943350 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:15:59.963457 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeNegativeTest-480692317-network', 'tenant_id': '003eb49c73ae4e6a962cbfb425b3c1eb'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:15:59.968556 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:15:59.972919 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Usage tracker for resource:network and tenant:003eb49c73ae4e6a962cbfb425b3c1eb is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:15:59.976911 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:15:59.983798 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:15:59.996214 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.010806 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Made reservation on behalf of 003eb49c73ae4e6a962cbfb425b3c1eb for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:00.013772 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.039582 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.059094 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:00.065049 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Usage tracker for resource:security_group_rule and tenant:003eb49c73ae4e6a962cbfb425b3c1eb is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:00.068931 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:00.076562 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:00.203390 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.211869 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.214795 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.218182 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.222731 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:00.263658 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] vlan segment allocate from pool success with {'vlan_id': 524, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:16:00.279402 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.279712 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Added segment b756bc07-d062-4b63-a043-c614b00fa2ca of type vlan for network bb3e0f6d-2046-4828-8f43-5f99ba5f3add May 24 11:16:00.282997 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:00.345136 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:00.352321 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Usage tracker for resource:security_group_rule and tenant:e65fade283614b028730cfea0f58bcad is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:00.362471 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b9070542-7975-4acb-bb57-8c4a7c17e6fc,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:00.377500 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:00.378429 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:00.391323 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.410986 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:00.425570 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:00.434154 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.441483 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=73805a20-c072-4ab0-bffb-f5df0cc148d2,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:00.453704 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:00.526325 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 0377cd0a-2401-44a9-a8d2-cc3ce593483b up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:00.598701 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.253s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:00.639105 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] vlan segment allocate from pool success with {'vlan_id': 516, 'physical_network': 'physnet1'} {{(pid=91820) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:16:00.686129 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.687242 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.segments_db [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Added segment 6b17b90a-a0d7-4ec4-b1f0-aca097bb9e65 of type vlan for network 22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0 May 24 11:16:00.705913 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.714937 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 0377cd0a-2401-44a9-a8d2-cc3ce593483b completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:00.756163 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.763729 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-684307', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-687011'] for router, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.765917 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-ceab4546-07f3-409b-8a5f-04dc80adcb49 None None] Network 1d92641a-d8f9-4ed6-9abd-f464bb26811d requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:16:00.962317 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.967837 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.975604 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 890 time: 1.8333862 May 24 11:16:00.975604 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:00.976041 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51294) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:00.976945 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:00.977782 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:01.004435 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] vlan segment allocate from pool success with {'vlan_id': 543, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:16:01.016733 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:01.017477 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Added segment b6a611a2-11cb-43b7-8194-22ac1ef4914b of type vlan for network 29272b37-7e2e-4e92-9978-bb32dde1813d May 24 11:16:01.041585 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.db.resource_extend [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] It took 0.18 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 24 11:16:01.046405 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Pushing event updated for resources: {'Network': ['ID=4fec6b68-64f1-4553-8ca5-f86c63ac9714,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:01.062214 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.608s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:01.068572 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:16:01.086034 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Request body: {'subnet': {'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714', 'cidr': '172.20.0.0/28', 'name': 'tempest-ServerRescueNegativeTestJSON-12783889-subnet', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:01.101670 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-772f8c5f-f7ca-4c88-b71b-c0a8886a5951 None None] Provisioning for port 0377cd0a-2401-44a9-a8d2-cc3ce593483b completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:01.102098 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:01.117433 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-772f8c5f-f7ca-4c88-b71b-c0a8886a5951 None None] Provisioning complete for port 0377cd0a-2401-44a9-a8d2-cc3ce593483b triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:01.118279 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-772f8c5f-f7ca-4c88-b71b-c0a8886a5951 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:01.123195 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:01.126769 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:01.127363 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:01.134601 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 1.7161019 May 24 11:16:01.138894 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:01.145562 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51304) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:01.160860 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:01.169140 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Usage tracker for resource:subnet and tenant:bf5fa11a013d4e1a8b86edc5fe49e97b is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:01.193719 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Request body: {'subnet': {'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeNegativeTest-480692317-subnet', 'tenant_id': '003eb49c73ae4e6a962cbfb425b3c1eb', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:01.203249 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:01.204973 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:01.213500 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Usage tracker for resource:subnet and tenant:003eb49c73ae4e6a962cbfb425b3c1eb is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:01.221799 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:01.236688 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:01.259945 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:01.279071 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Made reservation on behalf of 003eb49c73ae4e6a962cbfb425b3c1eb for: {'subnet': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:01.282905 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:01.466906 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Pushing event updated for resources: {'Network': ['ID=22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:01.473458 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 898 time: 2.5966909 May 24 11:16:01.482007 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.277s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:01.486920 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51312) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:01.506888 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Request body: {'subnet': {'network_id': '22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0', 'cidr': '172.20.0.0/28', 'name': 'tempest-EncryptedVolumesExtendAttachedTest-1271840068-subnet', 'tenant_id': 'de2d8ae871924896af66ae6c97b6ee1b', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:01.524714 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:01.534516 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Usage tracker for resource:subnet and tenant:de2d8ae871924896af66ae6c97b6ee1b is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:01.542514 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:01.553986 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network 22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:16:01.564688 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:01.604957 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Made reservation on behalf of de2d8ae871924896af66ae6c97b6ee1b for: {'subnet': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:01.608695 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:01.772746 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-772f8c5f-f7ca-4c88-b71b-c0a8886a5951 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:01.806419 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:01.810448 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=058e967f-9655-4f0a-9bd1-d825ec089aa1,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:01.829015 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.727s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:01.849066 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.854s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:02.052412 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Router f76568b3-17ef-487b-beeb-3f592a58e138 is scheduled to L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 with binding_index 1 {{(pid=91820) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 11:16:02.100165 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.db.resource_extend [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] It took 0.26 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 24 11:16:02.118228 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Made reservation on behalf of bf5fa11a013d4e1a8b86edc5fe49e97b for: {'subnet': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:02.120662 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Notify callbacks [] for subnet, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:02.148190 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-772f8c5f-f7ca-4c88-b71b-c0a8886a5951 None None] For port 0377cd0a-2401-44a9-a8d2-cc3ce593483b, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=0377cd0a-2401-44a9-a8d2-cc3ce593483b,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:02.160009 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-772f8c5f-f7ca-4c88-b71b-c0a8886a5951 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:02.181456 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:02.287566 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:02.296103 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:02.296672 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:02.360129 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-772f8c5f-f7ca-4c88-b71b-c0a8886a5951 None None] Pushing event updated for resources: {'Port': ['ID=0377cd0a-2401-44a9-a8d2-cc3ce593483b,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:02.369022 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-772f8c5f-f7ca-4c88-b71b-c0a8886a5951 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.187s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:02.377615 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Pushing event updated for resources: {'Network': ['ID=bb3e0f6d-2046-4828-8f43-5f99ba5f3add,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:02.387285 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.546s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:02.388750 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.080s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:02.403430 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network bb3e0f6d-2046-4828-8f43-5f99ba5f3add details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:16:02.529529 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 885 time: 3.1892879 May 24 11:16:02.538287 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51348) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:02.556337 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Request body: {'subnet': {'network_id': '29272b37-7e2e-4e92-9978-bb32dde1813d', 'cidr': '172.20.0.0/28', 'name': 'tempest-DeleteServersTestJSON-1595921365-subnet', 'tenant_id': 'e65fade283614b028730cfea0f58bcad', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:02.567864 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:02.582716 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Usage tracker for resource:subnet and tenant:e65fade283614b028730cfea0f58bcad is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:02.602705 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:02.620936 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:02.672761 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:16:02.691536 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Sync routers for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:16:02.696097 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Made reservation on behalf of e65fade283614b028730cfea0f58bcad for: {'subnet': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:02.700316 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:02.710795 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:16:02.712831 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-dd34365d-ed88-412c-967a-cd1bac92d02f None None] Provisioning complete for port 0377cd0a-2401-44a9-a8d2-cc3ce593483b triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:02.713352 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-dd34365d-ed88-412c-967a-cd1bac92d02f None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:02.714126 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:15:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:02.865051 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Pushing event updated for resources: {'Network': ['ID=29272b37-7e2e-4e92-9978-bb32dde1813d,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:02.872138 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.483s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:02.890341 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network 29272b37-7e2e-4e92-9978-bb32dde1813d details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:16:02.941183 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51358) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:03.179281 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:16:03.185899 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Notify callbacks [] for router, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:03.191083 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] 10.222.0.18 "POST /v2.0/routers HTTP/1.1" status: 201 len: 922 time: 5.4536538 May 24 11:16:03.197200 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51368) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:03.453900 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781'] for subnet, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:03.467825 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-27711c9c-51e6-4897-9df0-0b4d8eb5b9a1 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:03.480208 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3e57156d-ed54-4388-a38b-783ee396463a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:03.549896 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-27711c9c-51e6-4897-9df0-0b4d8eb5b9a1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:03.600463 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-27711c9c-51e6-4897-9df0-0b4d8eb5b9a1 admin admin] Usage tracker for resource:security_group_rule and tenant:f1366f50494d4dc2acce14425402cfc6 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:03.604130 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:15:59Z', 'revision_number': 3, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:15:58Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:03.608970 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Checking router: f76568b3-17ef-487b-beeb-3f592a58e138 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:16:03.609366 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Updating router f76568b3-17ef-487b-beeb-3f592a58e138 port 541360c9-23fa-4c8b-9a48-116e140aa6a4 binding host dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:16:03.621947 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-27711c9c-51e6-4897-9df0-0b4d8eb5b9a1 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:03.636689 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-27711c9c-51e6-4897-9df0-0b4d8eb5b9a1 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:03.660864 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781'] for subnet, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:03.668229 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781'] for subnet, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:03.676979 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:03.694866 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Pushing event updated for resources: {'Subnet': ['ID=1107b450-6c27-4f13-8b6e-abdbbbb1ea0f,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:03.697969 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.218s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:03.703457 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.015s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:03.727597 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-200b5a3c-fc9e-4abf-8099-72b49fa6c032 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:03.753111 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Checking segment: {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:03.753597 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Network hostable DHCP agents. Network: bb3e0f6d-2046-4828-8f43-5f99ba5f3add, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91819) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:16:03.904327 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Pushing event updated for resources: {'Subnet': ['ID=52267672-fd8c-45bd-89ea-d9d02a7b249a,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:03.907328 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Network bb3e0f6d-2046-4828-8f43-5f99ba5f3add is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91819) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:16:03.909768 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Checking segment: {'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:03.910113 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Network hostable DHCP agents. Network: 4fec6b68-64f1-4553-8ca5-f86c63ac9714, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91819) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:16:03.915142 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.212s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:03.934705 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, bb3e0f6d-2046-4828-8f43-5f99ba5f3add) {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:03.945902 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-a15e642c-1daa-49a7-aa78-38a9f64b89c9 None None] Network bb3e0f6d-2046-4828-8f43-5f99ba5f3add requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:16:03.949443 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:03.950030 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:03.953553 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Pushing event updated for resources: {'Subnet': ['ID=e290cdef-987a-4c48-922c-74e4d0a8adbf,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:03.971495 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.244s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:03.994853 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91819) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:16:04.023264 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, 4fec6b68-64f1-4553-8ca5-f86c63ac9714) {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:04.038698 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-2f9130f9-fb8e-41b5-a50a-250e2a7737c0 admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 851 time: 2.8786099 May 24 11:16:04.070210 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-27711c9c-51e6-4897-9df0-0b4d8eb5b9a1 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.079505 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:04.080647 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.088666 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-27711c9c-51e6-4897-9df0-0b4d8eb5b9a1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.136217 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:04.145194 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-27711c9c-51e6-4897-9df0-0b4d8eb5b9a1 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=f1366f50494d4dc2acce14425402cfc6&name=default HTTP/1.1" status: 200 len: 2975 time: 1.1941791 May 24 11:16:04.148426 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51404) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:04.180913 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-03fee488-dd44-451a-af6a-95e148c2590d admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 854 time: 3.1980715 May 24 11:16:04.187105 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51408) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:04.195587 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Request body: {'router': {'name': 'tempest-ServerRescueNegativeTestJSON-12783889-router', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:04.201357 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:04.206629 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Usage tracker for resource:router and tenant:bf5fa11a013d4e1a8b86edc5fe49e97b is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:04.210399 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:04.217259 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:04.226480 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-27711c9c-51e6-4897-9df0-0b4d8eb5b9a1 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c2d2baa8-1eed-4a4c-91d5-573b698b69f5,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:04.237839 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-684314', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4157'] for router, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.239098 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-684570', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-687045', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-715560', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4164'] for router, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.251060 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-27711c9c-51e6-4897-9df0-0b4d8eb5b9a1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.115s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:04.289035 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e387f792-abcd-4230-af05-c640e056bde8 None None] Network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:16:04.304571 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f023d0c0-e305-4995-9579-4f8bf676fb3f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.307769 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.340130 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Notify callbacks [] for port, precommit_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.358209 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.provisioning_blocks [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Transition to ACTIVE for port object 541360c9-23fa-4c8b-9a48-116e140aa6a4 will not be triggered until provisioned by entity L2. {{(pid=91822) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:16:04.374176 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.390914 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:04.460159 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 0377cd0a-2401-44a9-a8d2-cc3ce593483b up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:04.483128 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f023d0c0-e305-4995-9579-4f8bf676fb3f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.548167 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Attempting to bind port 541360c9-23fa-4c8b-9a48-116e140aa6a4 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:16:04.548601 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Attempting to bind port 541360c9-23fa-4c8b-9a48-116e140aa6a4 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:16:04.549352 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Attempting to bind port 541360c9-23fa-4c8b-9a48-116e140aa6a4 on network 858b8fad-a73f-4578-9079-6e731b9ff7e2 {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:04.562849 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 15, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:16:04.563577 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Checking segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91822) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:04.563783 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.564358 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Bound using segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:16:04.565018 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Bound port: 541360c9-23fa-4c8b-9a48-116e140aa6a4, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:16:04.565853 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.713579 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Pushing event updated for resources: {'Port': ['ID=541360c9-23fa-4c8b-9a48-116e140aa6a4,revision_number=2']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:04.723896 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:04.724247 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:04.724647 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:04.724974 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:04.725761 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:04.725761 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:04.726079 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:04.726321 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:04.726702 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:04.727022 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:04.727335 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:04.727644 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Tracked resource versions report: May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: pushed versions: May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: { 'AddressGroup': {'1.2'}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'ConntrackHelper': {'1.0'}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Log': {'1.0'}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Network': {'1.1'}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Port': {'1.7'}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'PortForwarding': {'1.2'}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'QosPolicy': {'1.8'}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'SecurityGroup': {'1.4'}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'SecurityGroupRule': {'1.2'}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'SubPort': {'1.0'}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Subnet': {'1.1'}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Trunk': {'1.1'}} May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: consumer versions: May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: { DHCP agent@dv-cinder-784092-9-iscsi: {}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: HyperV agent@hv-cinder-78409: {}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: L3 agent@dv-cinder-784092-9-iscsi: {}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: Metadata agent@dv-cinder-784092-9-iscsi: {}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: Metering agent@dv-cinder-784092-9-iscsi: {}, May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: Open vSwitch agent@dv-cinder-784092-9-iscsi: { 'AddressGroup': '1.2', May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'ConntrackHelper': '1.0', May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Log': '1.0', May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Network': '1.1', May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Port': '1.7', May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'PortForwarding': '1.2', May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'QosPolicy': '1.8', May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'SecurityGroup': '1.4', May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'SecurityGroupRule': '1.2', May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'SubPort': '1.0', May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Subnet': '1.1', May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: 'Trunk': '1.1'}} May 24 11:16:04.728917 dv-cinder-784092-9-iscsi neutron-server[91822]: {{(pid=91822) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 24 11:16:04.733473 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.343s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:04.744490 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] For port 541360c9-23fa-4c8b-9a48-116e140aa6a4, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91822) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:16:04.750857 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] For port 541360c9-23fa-4c8b-9a48-116e140aa6a4, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=541360c9-23fa-4c8b-9a48-116e140aa6a4,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91822) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:16:04.755947 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:04.762983 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:04.767152 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Checking segment: {'id': '6b17b90a-a0d7-4ec4-b1f0-aca097bb9e65', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 516, 'network_id': '22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:04.767700 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Network hostable DHCP agents. Network: 22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91820) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:16:04.828740 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-503b7896-6566-4a67-8391-b4e290fd93e3 None None] Network bb3e0f6d-2046-4828-8f43-5f99ba5f3add requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:16:04.841376 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Pushing event updated for resources: {'Port': ['ID=541360c9-23fa-4c8b-9a48-116e140aa6a4,revision_number=3']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:04.847191 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:04.874968 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:16:04.875727 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:15:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:04.927388 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51432) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:05.080439 dv-cinder-784092-9-iscsi neutron-server[91819]: WARNING keystonemiddleware.auth_token [-] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. May 24 11:16:05.108099 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Network 22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91820) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:16:05.121319 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 0377cd0a-2401-44a9-a8d2-cc3ce593483b triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:05.122085 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:05.140526 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:16:05.160296 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-80f576b0-c3fe-46c0-89ba-8d12e56cf5d8 None None] Network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:16:05.162732 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781'] for subnet, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:05.163669 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, 22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0) {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:05.186617 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:05.187629 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:05.195521 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b8d8612b-e2c2-4d08-a661-b5f572a85544 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:05.202901 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c2e055fd-e6ca-40a9-9c4d-96c8ced5256e None None] Network 22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:16:05.386079 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 862 time: 3.8980205 May 24 11:16:05.399353 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:05.442883 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Pushing event updated for resources: {'Subnet': ['ID=20bb60b1-11d7-4cba-b636-3bcdb98876e9,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:05.449523 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.254s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:05.502048 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Checking segment: {'id': 'b6a611a2-11cb-43b7-8194-22ac1ef4914b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 543, 'network_id': '29272b37-7e2e-4e92-9978-bb32dde1813d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:05.502522 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Network hostable DHCP agents. Network: 29272b37-7e2e-4e92-9978-bb32dde1813d, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91820) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:16:05.558875 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Network 29272b37-7e2e-4e92-9978-bb32dde1813d is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91820) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:16:05.600788 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, 29272b37-7e2e-4e92-9978-bb32dde1813d) {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:05.608861 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:05.610271 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:05.669142 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f023d0c0-e305-4995-9579-4f8bf676fb3f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:05.675821 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c96c99c9-5fe9-447e-bd42-8188b39c6e40 None None] Network 29272b37-7e2e-4e92-9978-bb32dde1813d requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:16:05.676302 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-687072', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-684328', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4178'] for router, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:05.702149 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:05.721154 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f023d0c0-e305-4995-9579-4f8bf676fb3f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c2d2baa8-1eed-4a4c-91d5-573b698b69f5,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:05.726974 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f023d0c0-e305-4995-9579-4f8bf676fb3f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:05.793306 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-14852e8c-e49e-42bc-a41e-8cdc36883e10 admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 849 time: 3.2502148 May 24 11:16:05.827298 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-acbcc8b1-826e-49e8-89c7-0b4218938391 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:05.832702 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-acbcc8b1-826e-49e8-89c7-0b4218938391 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Synchronizing usage tracker for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:network {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:05.871308 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f023d0c0-e305-4995-9579-4f8bf676fb3f admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:05.875355 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-f023d0c0-e305-4995-9579-4f8bf676fb3f admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/c2d2baa8-1eed-4a4c-91d5-573b698b69f5 HTTP/1.1" status: 204 len: 168 time: 1.7235503 May 24 11:16:05.889263 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-acbcc8b1-826e-49e8-89c7-0b4218938391 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Unset dirty status for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:network {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:05.890635 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-acbcc8b1-826e-49e8-89c7-0b4218938391 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/networks?id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add HTTP/1.1" status: 200 len: 814 time: 0.9616253 May 24 11:16:05.921778 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-14fe24c5-30f5-41c5-ac3f-eff3c79d6057 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/quotas/003eb49c73ae4e6a962cbfb425b3c1eb HTTP/1.1" status: 200 len: 375 time: 0.0200474 May 24 11:16:05.924962 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:15:59Z', 'revision_number': 3, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:04Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:05.945705 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b634477d-dc9b-413f-8f9f-ad35caa8f4ed None None] The sync data for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi is: [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:15:59Z', 'revision_number': 3, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:04Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:16:06.059422 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-7540bf1f-9b18-4405-b7de-ff222df381d8 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&fields=id HTTP/1.1" status: 200 len: 210 time: 0.1226628 May 24 11:16:06.097057 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51462) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:06.111261 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Allocated IP 172.20.0.1 (1d92641a-d8f9-4ed6-9abd-f464bb26811d/2fc4be2e-875c-4d10-b857-a48f90c53cf2/e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:16:06.488072 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:06.538599 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Transition to ACTIVE for port object e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 will not be triggered until provisioned by entity DHCP. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:16:06.552185 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:06.585596 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:06.597419 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:06.609819 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:06.730706 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Network 1d92641a-d8f9-4ed6-9abd-f464bb26811d is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:16:06.739567 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:06.748586 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:16:06.748868 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-687570'] for router_interface, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:06.805982 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Pushing event updated for resources: {'Port': ['ID=e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:06.814669 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.214s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:06.983699 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4a9cd060-9535-49ec-b4d2-8d5756101b9c admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.014578 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.055014 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 541360c9-23fa-4c8b-9a48-116e140aa6a4 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:07.070998 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Allocated IP 172.27.0.233 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/c9840383-de18-463c-8858-5536335f4466) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:16:07.132558 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e33b6761-f763-4146-9c03-a1c0496c0a5a None None] Network 22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:16:07.133454 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-4a9cd060-9535-49ec-b4d2-8d5756101b9c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:07.143320 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51486) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:07.163902 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-4a9cd060-9535-49ec-b4d2-8d5756101b9c admin admin] Usage tracker for resource:security_group_rule and tenant:78ced52951e748a09c55d2081a6c464e is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:07.168262 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-4a9cd060-9535-49ec-b4d2-8d5756101b9c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:07.175722 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-4a9cd060-9535-49ec-b4d2-8d5756101b9c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:07.210240 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-afc87d6c-bd2b-44aa-baa1-08bebb01e516 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.270135 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-afc87d6c-bd2b-44aa-baa1-08bebb01e516 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:07.274259 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4a9cd060-9535-49ec-b4d2-8d5756101b9c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.344237 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4a9cd060-9535-49ec-b4d2-8d5756101b9c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.360252 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:07.444434 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4a9cd060-9535-49ec-b4d2-8d5756101b9c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b62eca55-3059-44f1-b588-2d220a824a12,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:07.448735 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-42207b95-12ae-4e1e-8e61-31870df275c3 None None] Network 29272b37-7e2e-4e92-9978-bb32dde1813d requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:16:07.455325 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-4a9cd060-9535-49ec-b4d2-8d5756101b9c admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=78ced52951e748a09c55d2081a6c464e&name=default HTTP/1.1" status: 200 len: 2975 time: 1.3560274 May 24 11:16:07.462464 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4a9cd060-9535-49ec-b4d2-8d5756101b9c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:07.476311 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51494) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:07.582680 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-932d0cf6-98dd-40e1-ad67-ea64a352246d service neutron] http://10.222.0.18:9696/v2.0/extensions returned with HTTP 200 {{(pid=91819) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 11:16:07.583826 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-932d0cf6-98dd-40e1-ad67-ea64a352246d service neutron] 10.222.0.18 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.3077922 May 24 11:16:07.590238 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.597998 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.601070 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-afc87d6c-bd2b-44aa-baa1-08bebb01e516 admin admin] Usage tracker for resource:security_group_rule and tenant:1638d80b8eb84411878de94e243e3a24 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:07.606477 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.608408 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-afc87d6c-bd2b-44aa-baa1-08bebb01e516 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:07.610927 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-7229e28b-6e78-4686-9912-53e00d7df858 None None] Provisioning for port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:07.612717 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-533332', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-687487'] for router_interface, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.625520 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-7229e28b-6e78-4686-9912-53e00d7df858 None None] Provisioning complete for port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:07.625972 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-7229e28b-6e78-4686-9912-53e00d7df858 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.631839 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:07.783200 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-6acd465d-205b-45ca-ad4d-65f83b1d6c0f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.810830 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-7229e28b-6e78-4686-9912-53e00d7df858 None None] Port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:16:07.812905 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 541360c9-23fa-4c8b-9a48-116e140aa6a4 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:07.836816 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 541360c9-23fa-4c8b-9a48-116e140aa6a4 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:07.837481 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.896507 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-afc87d6c-bd2b-44aa-baa1-08bebb01e516 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:07.908810 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-6acd465d-205b-45ca-ad4d-65f83b1d6c0f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.909413 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Network 1d92641a-d8f9-4ed6-9abd-f464bb26811d is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:16:07.910412 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51512) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:07.922232 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:07.954469 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:07.963505 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-ac86b150-cfb1-48ad-84fb-0ecd3b21779d service neutron] 10.222.0.18 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3675902 May 24 11:16:07.971699 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51518) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:07.973285 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:08.015626 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Request body: {'network': {'name': 'tempest-EncryptedVolumesExtendAttachedTest-592589523-network', 'tenant_id': 'cb09a77682654da58923a77805e9b68a'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:08.060297 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:08.081980 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Usage tracker for resource:network and tenant:cb09a77682654da58923a77805e9b68a is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:08.091697 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:08.120574 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:16:08.178559 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:08.184753 dv-cinder-784092-9-iscsi neutron-server[91820]: WARNING keystonemiddleware.auth_token [-] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. May 24 11:16:08.188106 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:08.200431 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:16:08.201288 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Router f76568b3-17ef-487b-beeb-3f592a58e138 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:16:08.202659 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:08.221678 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 541360c9-23fa-4c8b-9a48-116e140aa6a4, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=541360c9-23fa-4c8b-9a48-116e140aa6a4,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:08.227397 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:08.234317 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:08.267358 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Pushing event updated for resources: {'Port': ['ID=c9840383-de18-463c-8858-5536335f4466,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:08.301589 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-e9d34ab8-e6dc-4394-8dfb-f5f047767338 None None] Provisioning complete for port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:08.301964 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e9d34ab8-e6dc-4394-8dfb-f5f047767338 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:08.328490 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Pushing event updated for resources: {'Port': ['ID=e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:08.340184 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.367s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:08.343557 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.709s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:08.355501 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Made reservation on behalf of cb09a77682654da58923a77805e9b68a for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:08.358576 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:08.373974 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=541360c9-23fa-4c8b-9a48-116e140aa6a4,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:08.379040 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.145s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:08.379801 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-e9d34ab8-e6dc-4394-8dfb-f5f047767338 None None] Port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:16:08.381899 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:08.438530 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-6acd465d-205b-45ca-ad4d-65f83b1d6c0f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:08.471037 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4a9cd060-9535-49ec-b4d2-8d5756101b9c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:08.547164 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:08.562150 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Usage tracker for resource:security_group_rule and tenant:cb09a77682654da58923a77805e9b68a is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:08.565281 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6acd465d-205b-45ca-ad4d-65f83b1d6c0f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b62eca55-3059-44f1-b588-2d220a824a12,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:08.570663 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:08.580332 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-6acd465d-205b-45ca-ad4d-65f83b1d6c0f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:08.582891 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-a911c42b-3116-4e0d-a3cb-d9cd360bc896 service neutron] 10.222.0.18 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.5605001 May 24 11:16:08.590017 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:08.666584 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-6acd465d-205b-45ca-ad4d-65f83b1d6c0f admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:08.669804 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-6acd465d-205b-45ca-ad4d-65f83b1d6c0f admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/b62eca55-3059-44f1-b588-2d220a824a12 HTTP/1.1" status: 204 len: 168 time: 1.1921027 May 24 11:16:08.730585 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-840e8f6a-2fcb-4041-bafe-8f4de9c15356 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:08.745130 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-840e8f6a-2fcb-4041-bafe-8f4de9c15356 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/networks?id=29272b37-7e2e-4e92-9978-bb32dde1813d HTTP/1.1" status: 200 len: 812 time: 0.8036733 May 24 11:16:08.775749 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-808d87cf-795b-49ce-9eb2-239759aa875d tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/quotas/e65fade283614b028730cfea0f58bcad HTTP/1.1" status: 200 len: 375 time: 0.0302916 May 24 11:16:08.814195 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 541360c9-23fa-4c8b-9a48-116e140aa6a4 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:08.934886 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-b6fe2692-1938-421e-b49d-b3c6a4c671b5 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=e65fade283614b028730cfea0f58bcad&fields=id HTTP/1.1" status: 200 len: 210 time: 0.1608267 May 24 11:16:08.990134 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:09.009555 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 541360c9-23fa-4c8b-9a48-116e140aa6a4 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:09.009903 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:09.018345 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-afc87d6c-bd2b-44aa-baa1-08bebb01e516 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:09.030375 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:16:09.033219 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-afc87d6c-bd2b-44aa-baa1-08bebb01e516 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:09.048678 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Sync routers for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:16:09.065282 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:09.085900 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] 10.222.0.18 "PUT /v2.0/routers/f76568b3-17ef-487b-beeb-3f592a58e138/add_router_interface HTTP/1.1" status: 200 len: 503 time: 5.8778529 May 24 11:16:09.112086 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-afc87d6c-bd2b-44aa-baa1-08bebb01e516 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=1638d80b8eb84411878de94e243e3a24&name=default HTTP/1.1" status: 200 len: 2975 time: 1.9661078 May 24 11:16:09.118109 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51548) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:09.120821 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-afc87d6c-bd2b-44aa-baa1-08bebb01e516 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2366e792-fcf0-419c-9847-bb0e6e30ee45,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:09.152979 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:16:09.153998 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:15:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:09.167115 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-afc87d6c-bd2b-44aa-baa1-08bebb01e516 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:09.230626 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-aea1e32f-280e-4478-b80b-371f3e4c8008 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:09.293138 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-aea1e32f-280e-4478-b80b-371f3e4c8008 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:09.307785 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:09.425259 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:09.439315 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-aea1e32f-280e-4478-b80b-371f3e4c8008 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:09.444529 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:09.460586 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aea1e32f-280e-4478-b80b-371f3e4c8008 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2366e792-fcf0-419c-9847-bb0e6e30ee45,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:09.466846 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-caab278d-b114-40e6-8ff9-5e5d3f875908 req-6cd11ce7-d449-4730-8db3-a38a4c50a631 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 210 time: 0.1677558 May 24 11:16:09.471370 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:09.478448 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-aea1e32f-280e-4478-b80b-371f3e4c8008 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:09.486351 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-aea1e32f-280e-4478-b80b-371f3e4c8008 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:09.490715 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-aea1e32f-280e-4478-b80b-371f3e4c8008 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/2366e792-fcf0-419c-9847-bb0e6e30ee45 HTTP/1.1" status: 204 len: 168 time: 0.3679049 May 24 11:16:09.523798 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:16:09.539287 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b284fa57-77ac-481f-a5db-b2621c00144b,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:09.569592 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:09.607251 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] vlan segment allocate from pool success with {'vlan_id': 514, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:16:09.640661 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:09.641324 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Added segment be60d73a-b842-4a9f-ae55-b20e1ec8ab90 of type vlan for network c731a0f0-19d8-4a2f-8776-6ddcb26d147c May 24 11:16:09.664330 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-2c4b57a6-5ff2-478c-8159-bfbec2fc1853 service neutron] http://10.222.0.18:9696/v2.0/extensions returned with HTTP 200 {{(pid=91820) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 11:16:09.671352 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-2c4b57a6-5ff2-478c-8159-bfbec2fc1853 service neutron] 10.222.0.18 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.3807511 May 24 11:16:09.975621 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-5396f5fa-728d-4bcd-916c-1151f2b970b3 service neutron] 10.222.0.18 "GET /v2.0/networks/29272b37-7e2e-4e92-9978-bb32dde1813d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3033025 May 24 11:16:09.978081 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-684307', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-687011'] for router, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:10.088672 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:10.257867 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-f10bf4e9-6bef-45e6-a899-6d430fa191bf service neutron] 10.222.0.18 "GET /v2.0/networks/29272b37-7e2e-4e92-9978-bb32dde1813d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2490599 May 24 11:16:10.277401 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:10.289827 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:10.290270 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:10.301780 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:10.463164 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:07Z', 'revision_number': 2, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:10.463478 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 897 time: 2.4531400 May 24 11:16:10.463914 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51596) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:10.470150 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Request body: {'subnet': {'network_id': 'c731a0f0-19d8-4a2f-8776-6ddcb26d147c', 'cidr': '172.20.0.0/28', 'name': 'tempest-EncryptedVolumesExtendAttachedTest-592589523-subnet', 'tenant_id': 'cb09a77682654da58923a77805e9b68a', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:10.478089 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:10.479002 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Checking router: f76568b3-17ef-487b-beeb-3f592a58e138 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:16:10.479704 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Updating router f76568b3-17ef-487b-beeb-3f592a58e138 port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 binding host dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:16:10.488243 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Usage tracker for resource:subnet and tenant:cb09a77682654da58923a77805e9b68a is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:10.495291 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Pushing event updated for resources: {'Network': ['ID=c731a0f0-19d8-4a2f-8776-6ddcb26d147c,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:10.498966 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:10.502072 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.201s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:10.507439 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:10.532143 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Made reservation on behalf of cb09a77682654da58923a77805e9b68a for: {'subnet': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:10.532384 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Router 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 is scheduled to L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 with binding_index 1 {{(pid=91819) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 11:16:10.568405 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:10.569050 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Notify callbacks [] for subnet, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:10.575603 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51606) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:10.585611 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network c731a0f0-19d8-4a2f-8776-6ddcb26d147c details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:16:10.600535 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Request body: {'security_group': {'name': 'tempest-securitygroup--618302121', 'description': 'tempest-description--331832526'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:10.921470 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:10.945885 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-09a0d2fd-ea12-40e8-aefe-1788ac2709ea req-5e8fe45d-f176-438c-a536-954bf41dc118 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 210 time: 0.2126052 May 24 11:16:11.055349 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Usage tracker for resource:security_group and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:11.095687 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:11.109084 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:11.113850 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a55570d3-cccd-4f9d-ba3a-f12673c3e6c8 req-cf366097-299c-40d0-8cf8-4115be9a727e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 210 time: 0.3691123 May 24 11:16:11.133928 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:11.185878 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:11.192031 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Usage tracker for resource:security_group_rule and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:11.192961 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:11.202904 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:11.205797 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:16:11.217754 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Sync routers for ids ['24e49d59-ce93-4246-ae1a-cf4080a4b8f5'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:16:11.242378 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Routers {'24e49d59-ce93-4246-ae1a-cf4080a4b8f5'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:16:11.245523 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:15:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['24e49d59-ce93-4246-ae1a-cf4080a4b8f5'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:11.319185 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Notify callbacks [] for router, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:11.329163 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] 10.222.0.18 "POST /v2.0/routers HTTP/1.1" status: 201 len: 929 time: 7.1403048 May 24 11:16:11.340106 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51652) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:11.800074 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Notify callbacks [] for port, precommit_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:11.845686 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Transition to ACTIVE for port object e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:16:11.857421 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:11.872947 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Set router 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:16:11.894449 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:12.021790 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781'] for subnet, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:12.034124 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0cf02d72-9c89-4e6e-9933-ac81d8b1d915 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:12.256191 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:12.280202 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Pushing event updated for resources: {'Subnet': ['ID=994fc348-a3fc-4763-80fe-d1d9e2dc377a,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:12.297901 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.258s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:12.435609 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cad21629-ca3a-4ef6-a7e1-0c7fb7a71983 req-7fcca146-b3a0-40f1-80db-bd97648bd883 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 210 time: 0.1630206 May 24 11:16:12.492384 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Checking segment: {'id': 'be60d73a-b842-4a9f-ae55-b20e1ec8ab90', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 514, 'network_id': 'c731a0f0-19d8-4a2f-8776-6ddcb26d147c'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:12.492739 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Network hostable DHCP agents. Network: c731a0f0-19d8-4a2f-8776-6ddcb26d147c, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91820) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:16:12.513772 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Network c731a0f0-19d8-4a2f-8776-6ddcb26d147c is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91820) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:16:12.526690 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, c731a0f0-19d8-4a2f-8776-6ddcb26d147c) {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:12.534747 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:12.537372 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:12.580515 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-6c292cb1-bae0-4558-a348-3ab8c994dfc7 None None] Network c731a0f0-19d8-4a2f-8776-6ddcb26d147c requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:16:12.592348 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 861 time: 2.1273730 May 24 11:16:12.649897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:12.752349 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:12.840713 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c9771ac0-1c52-4756-978f-810940b6daed None None] Pushing event updated for resources: {'Port': ['ID=e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:12.843358 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'name': 'tempest-ServerRescueNegativeTestJSON-12783889-router', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233'}], 'enable_snat': True}, 'gw_port_id': 'c9840383-de18-463c-8858-5536335f4466', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:16:04Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 3, 'project_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'gw_port': {'id': 'c9840383-de18-463c-8858-5536335f4466', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:07:5a:29', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 81, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:06Z', 'updated_at': '2021-05-24T11:16:07Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:12.854743 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Checking router: 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:16:12.855127 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Updating router 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 port c9840383-de18-463c-8858-5536335f4466 binding host dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:16:12.861031 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c9771ac0-1c52-4756-978f-810940b6daed None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.967s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:12.879035 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Attempting to bind port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:16:12.887628 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Attempting to bind port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:16:12.888267 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Attempting to bind port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 on network 1d92641a-d8f9-4ed6-9abd-f464bb26811d {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:12.918234 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 15, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:16:12.922165 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Checking segment: {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:12.924139 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:12.925696 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Bound using segment: {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:16:12.926226 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Bound port: e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:16:12.926620 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:12.975260 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:13.019056 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-206103bd-0fb2-4150-b55e-c10f6b63f925 req-2618f4cf-26d3-480a-ba91-4a855b872f96 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 210 time: 0.2250848 May 24 11:16:13.057674 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event updated for resources: {'SecurityGroup': ['ID=20a7f42b-2874-40d1-94c8-66d210ae913e,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:13.063656 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.321s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:13.086352 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:13.091181 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51730) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:13.093402 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1778 time: 2.5132437 May 24 11:16:13.098224 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Request body: {'security_group_rule': {'security_group_id': '20a7f42b-2874-40d1-94c8-66d210ae913e', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:13.119988 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:13.140557 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Usage tracker for resource:security_group_rule and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:13.145976 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:13.173552 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51734) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:13.179888 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:13.308228 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] For port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:16:13.324513 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-2c62dbdf-1e4c-469e-b034-0040781ed027 None None] Network c731a0f0-19d8-4a2f-8776-6ddcb26d147c requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:16:13.389614 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:13.468547 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_v2 [None req-0935c80e-1e33-4112-83af-e538e7786c5d admin admin] Deleting subnet e290cdef-987a-4c48-922c-74e4d0a8adbf {{(pid=91820) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:16:13.505773 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0935c80e-1e33-4112-83af-e538e7786c5d admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:13.561434 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] For port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:16:13.602588 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:13.620412 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:13.735393 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.93', 65195) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:13.809663 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0935c80e-1e33-4112-83af-e538e7786c5d admin admin] Notify callbacks [] for subnet, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:13.910949 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0935c80e-1e33-4112-83af-e538e7786c5d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:13.914639 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Notify callbacks [] for port, precommit_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:13.952520 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.provisioning_blocks [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Transition to ACTIVE for port object c9840383-de18-463c-8858-5536335f4466 will not be triggered until provisioned by entity L2. {{(pid=91822) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:16:13.970971 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:13.975200 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:14.006691 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-eb35e23c-73fc-46cf-8e33-7b8ff9e3fd62 req-c48b12ba-7bcf-4daf-93d0-4d837364e232 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 210 time: 0.1443999 May 24 11:16:14.071670 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:16:14.072136 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:15:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:14.074520 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:14.117995 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:14.230972 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:14.307055 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-099a4a1a-0c7d-4e78-b27a-03391453046d None None] Pushing event updated for resources: {'Port': ['ID=c9840383-de18-463c-8858-5536335f4466,revision_number=2']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:14.341667 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Attempting to bind port c9840383-de18-463c-8858-5536335f4466 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:16:14.344595 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Attempting to bind port c9840383-de18-463c-8858-5536335f4466 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:16:14.345047 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Attempting to bind port c9840383-de18-463c-8858-5536335f4466 on network 858b8fad-a73f-4578-9079-6e731b9ff7e2 {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:14.350606 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-099a4a1a-0c7d-4e78-b27a-03391453046d None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.375s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:14.374920 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 15, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:16:14.375311 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Checking segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91822) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:14.375901 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:14.376315 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Bound using segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:16:14.378371 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Bound port: c9840383-de18-463c-8858-5536335f4466, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:16:14.378804 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:14.494455 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.93', 65201) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:14.605878 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-905056c5-5a06-470f-8ccf-24d66134e2fb req-3d040129-7259-45d8-9d1a-3e810d0c067c tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 210 time: 0.1047943 May 24 11:16:14.611601 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] For port c9840383-de18-463c-8858-5536335f4466, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91822) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:16:14.615668 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] For port c9840383-de18-463c-8858-5536335f4466, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=c9840383-de18-463c-8858-5536335f4466,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91822) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:16:14.621793 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:14.641435 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:14.694339 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:14.796757 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=f33ade18-8549-4c42-b768-e6580166bef5,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:14.804067 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Routers {'24e49d59-ce93-4246-ae1a-cf4080a4b8f5'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:16:14.804723 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:15:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['24e49d59-ce93-4246-ae1a-cf4080a4b8f5'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:14.823807 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c9771ac0-1c52-4756-978f-810940b6daed None None] Pushing event updated for resources: {'Port': ['ID=e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:14.823952 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.125s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:14.841947 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c9771ac0-1c52-4756-978f-810940b6daed None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 1.221s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:14.890331 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:14.895936 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 1.8034968 May 24 11:16:14.920577 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51802) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:14.976366 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Request body: {'security_group_rule': {'security_group_id': '20a7f42b-2874-40d1-94c8-66d210ae913e', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:14.994998 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:16:15.025397 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:15.049375 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Allocated IP 172.20.0.1 (4fec6b68-64f1-4553-8ca5-f86c63ac9714/52267672-fd8c-45bd-89ea-d9d02a7b249a/97fb6682-0049-400a-84fe-731434f40cc6) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:16:15.051223 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Usage tracker for resource:security_group_rule and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:15.060644 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:15.100430 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:15.176779 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-2f8f9884-60a8-4806-821c-5ba9cbf09dfb tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:15.180221 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-099a4a1a-0c7d-4e78-b27a-03391453046d None None] Pushing event updated for resources: {'Port': ['ID=c9840383-de18-463c-8858-5536335f4466,revision_number=3']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:15.181429 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-2f8f9884-60a8-4806-821c-5ba9cbf09dfb tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "GET /v2.0/networks?id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add HTTP/1.1" status: 200 len: 814 time: 1.4330108 May 24 11:16:15.223650 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-099a4a1a-0c7d-4e78-b27a-03391453046d None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.582s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:15.349940 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] Set router 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:16:15.367974 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Request body: {'port': {'device_id': 'fd3abcad-fe0f-460b-b250-becb3ccb2624', 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add', 'admin_state_up': True, 'tenant_id': '003eb49c73ae4e6a962cbfb425b3c1eb'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:15.456419 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-61c9739c-ed8a-4c56-8f64-ac9d71bd7438 req-5ee08565-c03a-4a8b-8dc8-03523a1858f6 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 210 time: 0.0696228 May 24 11:16:15.469560 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:15.550364 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:15.589580 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:15.681824 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:16.040217 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:16:16.114086 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:13Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:16.134727 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ca0203d4-437c-4fc6-a9c2-31cfc63604ff None None] The sync data for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi is: [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:13Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:16:16.170844 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:16.185704 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0935c80e-1e33-4112-83af-e538e7786c5d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781'] for subnet, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:16.188412 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:16.206439 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:16.207425 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Usage tracker for resource:port and tenant:003eb49c73ae4e6a962cbfb425b3c1eb is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:16.211555 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:16.292300 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Quota usage for port was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:16.297266 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:16.333301 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:16.344160 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0935c80e-1e33-4112-83af-e538e7786c5d admin admin] Pushing event deleted for resources: {'Subnet': ['ID=e290cdef-987a-4c48-922c-74e4d0a8adbf,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:16.389661 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0935c80e-1e33-4112-83af-e538e7786c5d admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:16.440362 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0935c80e-1e33-4112-83af-e538e7786c5d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.228s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:16.443569 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.109s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:16.473785 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-0935c80e-1e33-4112-83af-e538e7786c5d admin admin] 10.222.0.18 "DELETE /v2.0/subnets/e290cdef-987a-4c48-922c-74e4d0a8adbf HTTP/1.1" status: 204 len: 168 time: 3.2976444 May 24 11:16:16.483402 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Made reservation on behalf of 003eb49c73ae4e6a962cbfb425b3c1eb for: {'port': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:16.491822 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:16.516777 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51850) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:16.528089 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:16.541078 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 1.5883410 May 24 11:16:16.551676 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51856) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:16.558356 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=b95e0f77-013a-4e54-9818-76ceff4a472a,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:16.564832 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.123s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:16.622568 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'name': 'tempest-ServerRescueNegativeTestJSON-12783889-router', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233'}], 'enable_snat': True}, 'gw_port_id': 'c9840383-de18-463c-8858-5536335f4466', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:16:04Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 3, 'project_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'gw_port': {'id': 'c9840383-de18-463c-8858-5536335f4466', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:07:5a:29', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 81, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:06Z', 'updated_at': '2021-05-24T11:16:14Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:16.635532 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ed04772b-fe1d-4ea0-bb77-e54af8846890 None None] The sync data for ids ['24e49d59-ce93-4246-ae1a-cf4080a4b8f5'] in dv-cinder-784092-9-iscsi is: [{'id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'name': 'tempest-ServerRescueNegativeTestJSON-12783889-router', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233'}], 'enable_snat': True}, 'gw_port_id': 'c9840383-de18-463c-8858-5536335f4466', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:16:04Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 3, 'project_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'gw_port': {'id': 'c9840383-de18-463c-8858-5536335f4466', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:07:5a:29', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 81, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:06Z', 'updated_at': '2021-05-24T11:16:14Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:16:16.637978 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Request body: {'floatingip': {'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:16.765578 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:16:16.766143 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-687570'] for router_interface, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:16.837321 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:16.905474 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Usage tracker for resource:floatingip and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:16.914555 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:16.981606 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:16.989104 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51868) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:17.053696 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:17.067423 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:17.069044 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:17.178803 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5a7a2267-aa13-4a43-ab2f-0ee65712d1ea req-42709349-688b-48bb-ac1d-0b576c6f3022 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 210 time: 0.3303175 May 24 11:16:17.268256 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:17.353057 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:17.405699 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:17.450264 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Pushing event updated for resources: {'Port': ['ID=97fb6682-0049-400a-84fe-731434f40cc6,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:17.518375 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 1.929s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:17.578824 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-84e8c7ab-8260-40c1-aea1-37b760978b6d tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:17.582311 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-84e8c7ab-8260-40c1-aea1-37b760978b6d tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Synchronizing usage tracker for tenant:e65fade283614b028730cfea0f58bcad on resource:network {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:17.686714 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-eb785d7a-1d80-4481-80ed-d07e680abc86 req-b524f8f5-1b8b-4e22-9282-1ef304766a30 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 210 time: 1.7225516 May 24 11:16:17.695516 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:17.717838 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-84e8c7ab-8260-40c1-aea1-37b760978b6d tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Unset dirty status for tenant:e65fade283614b028730cfea0f58bcad on resource:network {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:17.717838 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-84e8c7ab-8260-40c1-aea1-37b760978b6d tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.93 "GET /v2.0/networks?id=29272b37-7e2e-4e92-9978-bb32dde1813d HTTP/1.1" status: 200 len: 812 time: 3.2067101 May 24 11:16:17.738384 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Request body: {'port': {'device_id': '82ca8978-fbb3-4792-8bfd-f8b2e565b925', 'network_id': '29272b37-7e2e-4e92-9978-bb32dde1813d', 'admin_state_up': True, 'tenant_id': 'e65fade283614b028730cfea0f58bcad'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:17.915725 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:17.925784 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:17.941829 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b3fa6816-0b68-48a8-920e-d32d0cfdb628 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:18.039460 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51884) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:18.142647 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:18.163896 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.222s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:18.338162 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:18.375308 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > segment before_delete 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.376410 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > subnet precommit_delete_associations 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.378104 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.378928 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > port after_create 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.379592 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > port after_delete 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.380242 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > port after_update 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.381192 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > segment after_delete 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.381544 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > subnet after_create 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.381991 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.382492 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > subnet after_update 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.504254 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:18.552321 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:18.612439 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Usage tracker for resource:port and tenant:e65fade283614b028730cfea0f58bcad is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:18.639939 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Quota usage for port was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:18.659733 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:18.702317 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > subnet after_create 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.703040 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.703618 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > subnet before_create 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.704169 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Subscribe: > subnet before_update 55550000 {{(pid=91820) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:18.704599 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0']}} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:18.739322 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '6b17b90a-a0d7-4ec4-b1f0-aca097bb9e65') {'for_net_delete': True} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:18.739322 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '6b17b90a-a0d7-4ec4-b1f0-aca097bb9e65') {} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:18.766789 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534861936'] for segment, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:18.798522 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:18.811848 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Made reservation on behalf of e65fade283614b028730cfea0f58bcad for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:18.820989 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Releasing vlan 516 on physical network physnet1 to pool {{(pid=91820) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:16:18.821862 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535262766'] for segment, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:18.823811 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:19.056370 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device c9840383-de18-463c-8858-5536335f4466 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: from cryptography.utils import int_from_bytes May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: from cryptography.utils import int_from_bytes May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: Traceback (most recent call last): May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: timer() May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: cb(*args, **kw) May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: return func(*args, **kwargs) May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: self._notify() May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: self.callback(batched_events) May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: event.method(event) May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: return f(self, *a, **k) May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._get(url).json() May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: return self.request(url, 'GET', **kwargs) May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:16:19.069826 dv-cinder-784092-9-iscsi neutron-server[91820]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:16:19.108832 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:19.257824 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port c9840383-de18-463c-8858-5536335f4466 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:19.270950 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port c9840383-de18-463c-8858-5536335f4466 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:19.271592 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:19.287948 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:16:19.454899 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:19.465872 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b73cb5ef-4254-4da9-8e5d-04b99c198d13 req-d17630c3-ff3c-4640-b0b6-cd98f0d06446 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 210 time: 0.2543638 May 24 11:16:19.471630 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:19.516402 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-af29ed40-83d6-409c-9d17-5d21c28cd8d8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:19.575286 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-660d722d-5185-42d4-9536-6d4f6f6ae3a0 req-9c21fc26-02df-4916-ad34-9fdbc6de39bd tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 210 time: 0.5099678 May 24 11:16:19.585449 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] 10.222.0.18 "DELETE /v2.0/networks/22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0 HTTP/1.1" status: 204 len: 168 time: 3.0604599 May 24 11:16:19.597395 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:19.598544 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51906) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:19.605132 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Pushing event deleted for resources: {'Network': ['ID=22d3ee57-05c4-47c8-8ad5-d80b8c4e86d0,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:19.665201 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Allocated IP 172.20.0.8 (bb3e0f6d-2046-4828-8f43-5f99ba5f3add/1107b450-6c27-4f13-8b6e-abdbbbb1ea0f/45911d6f-9595-40c5-8b29-75259c824bfe) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:16:19.674100 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-af29ed40-83d6-409c-9d17-5d21c28cd8d8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:19.685807 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-af29ed40-83d6-409c-9d17-5d21c28cd8d8 admin admin] Usage tracker for resource:security_group_rule and tenant:161e971159e84c1fbe58007a655782f9 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:19.688601 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port c9840383-de18-463c-8858-5536335f4466, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=c9840383-de18-463c-8858-5536335f4466,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:19.689450 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-af29ed40-83d6-409c-9d17-5d21c28cd8d8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:19.695898 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:19.698782 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-af29ed40-83d6-409c-9d17-5d21c28cd8d8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:19.699245 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:19.704945 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.234s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:19.805586 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Allocated IP 172.20.0.5 (29272b37-7e2e-4e92-9978-bb32dde1813d/20bb60b1-11d7-4cba-b636-3bcdb98876e9/65acda30-2128-434e-a144-f18a69777754) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:16:19.820286 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_v2 [None req-1446dcc6-3038-49c6-8a2f-a34364600b9b admin admin] Deleting subnet 994fc348-a3fc-4763-80fe-d1d9e2dc377a {{(pid=91820) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:16:19.839632 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1446dcc6-3038-49c6-8a2f-a34364600b9b admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:19.865024 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:19.878765 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:19.880852 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:19.886150 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-533332', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-687487'] for router_interface, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:19.906631 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c7bb35a9-7fe2-4888-8cf1-85e6d551f097 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:19.953127 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1446dcc6-3038-49c6-8a2f-a34364600b9b admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534861947'] for subnet, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:20.048076 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:20.048471 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:20.159775 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=c9840383-de18-463c-8858-5536335f4466,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:20.166631 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-46e0c73f-e3d3-409b-af58-86fb297a947e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:20.182281 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.303s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:20.218653 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-46e0c73f-e3d3-409b-af58-86fb297a947e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:20.223668 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-46e0c73f-e3d3-409b-af58-86fb297a947e admin admin] Usage tracker for resource:security_group_rule and tenant:f87b73be6af741b2902bb3f8bebb4828 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:20.227322 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-46e0c73f-e3d3-409b-af58-86fb297a947e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:20.238137 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-46e0c73f-e3d3-409b-af58-86fb297a947e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:20.366976 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Pushing event updated for resources: {'Port': ['ID=97fb6682-0049-400a-84fe-731434f40cc6,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:20.370545 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.464s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:20.384572 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-46e0c73f-e3d3-409b-af58-86fb297a947e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:20.429551 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-46e0c73f-e3d3-409b-af58-86fb297a947e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:20.484855 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-aea1e32f-280e-4478-b80b-371f3e4c8008 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:20.536455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1446dcc6-3038-49c6-8a2f-a34364600b9b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:20.738365 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:16:20.797226 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:20.850424 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device c9840383-de18-463c-8858-5536335f4466 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:20.973564 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:20.993504 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-eb79afb6-7ce5-40ca-a93e-d4f17580de8f req-ab551e12-547a-4afe-a08e-9d3c5fcf9512 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 210 time: 0.2729030 May 24 11:16:20.996990 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port c9840383-de18-463c-8858-5536335f4466 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:20.997458 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:21.109750 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-46e0c73f-e3d3-409b-af58-86fb297a947e admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=f87b73be6af741b2902bb3f8bebb4828&name=default HTTP/1.1" status: 200 len: 2979 time: 2.0719292 May 24 11:16:21.112521 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 51922) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:21.153329 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-18ab9d7d-6667-4b60-8175-379dac6694ce req-ec4a1fe1-7abc-4196-a13f-2bfccbe74f66 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 210 time: 0.2903447 May 24 11:16:21.255467 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-46e0c73f-e3d3-409b-af58-86fb297a947e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=af222a7b-cca6-4765-946f-530d26ea6c1a,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:21.262820 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-46e0c73f-e3d3-409b-af58-86fb297a947e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.778s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:21.270101 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:21.348614 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:21.366785 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8c05785e-a692-4825-b630-470c4b23529c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:21.393022 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:21.424953 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Router 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:16:21.469503 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Network 29272b37-7e2e-4e92-9978-bb32dde1813d is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:16:21.470711 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:21.474161 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:16:21.482037 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:21.482934 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:21.585584 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 3.8638868 May 24 11:16:21.592284 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Pushing event updated for resources: {'Port': ['ID=65acda30-2128-434e-a144-f18a69777754,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:21.596731 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-af29ed40-83d6-409c-9d17-5d21c28cd8d8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:21.663310 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.297s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:21.715321 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-af29ed40-83d6-409c-9d17-5d21c28cd8d8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:21.730596 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:21.739962 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:16:21.747749 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] 10.222.0.18 "PUT /v2.0/routers/24e49d59-ce93-4246-ae1a-cf4080a4b8f5/add_router_interface HTTP/1.1" status: 200 len: 503 time: 10.3981483 May 24 11:16:21.762595 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Sync routers for ids ['24e49d59-ce93-4246-ae1a-cf4080a4b8f5'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:16:21.766061 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-afc87d6c-bd2b-44aa-baa1-08bebb01e516 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:21.790071 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Routers {'24e49d59-ce93-4246-ae1a-cf4080a4b8f5'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:16:21.790743 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:15:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['24e49d59-ce93-4246-ae1a-cf4080a4b8f5'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:21.922977 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af29ed40-83d6-409c-9d17-5d21c28cd8d8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=df45a7c4-d00d-43d1-a1aa-ebe6a351596e,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:21.947098 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cc02d756-3752-4aa2-8cb5-4037f6c443b6 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:21.963577 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-af29ed40-83d6-409c-9d17-5d21c28cd8d8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.197s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:21.979226 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-af29ed40-83d6-409c-9d17-5d21c28cd8d8 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=161e971159e84c1fbe58007a655782f9&name=default HTTP/1.1" status: 200 len: 2979 time: 4.8174441 May 24 11:16:21.983986 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51936) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:22.084823 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cc02d756-3752-4aa2-8cb5-4037f6c443b6 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:22.115321 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Allocated IP 172.27.0.86 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/777aecd4-bf5c-4efa-a7e0-b3f488ba709c) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:16:22.143858 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1446dcc6-3038-49c6-8a2f-a34364600b9b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534788488', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535456750'] for subnet, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:22.172615 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:22.178079 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:22.237403 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1446dcc6-3038-49c6-8a2f-a34364600b9b admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:22.286082 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-250de9b1-b2f1-4647-9ed1-82be32d9501a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:22.363661 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1446dcc6-3038-49c6-8a2f-a34364600b9b admin admin] Pushing event deleted for resources: {'Subnet': ['ID=994fc348-a3fc-4763-80fe-d1d9e2dc377a,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:22.436917 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1446dcc6-3038-49c6-8a2f-a34364600b9b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.259s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:22.438007 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.016s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:22.504001 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Set router 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:16:22.608415 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-a8d1615a-f13b-4ae9-83b6-6d6060ea9f9b tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] http://10.222.0.18:9696/v2.0/extensions returned with HTTP 200 {{(pid=91819) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 11:16:22.617235 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-a8d1615a-f13b-4ae9-83b6-6d6060ea9f9b tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.93 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0209856 May 24 11:16:22.628884 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-250de9b1-b2f1-4647-9ed1-82be32d9501a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:22.694344 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Network bb3e0f6d-2046-4828-8f43-5f99ba5f3add is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:16:22.733449 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:22.763165 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:16:22.869895 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-be0eb637-601b-44fb-bcb7-985621098816 req-8016101d-a037-4389-bcc6-2ffce4db0b3a tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:22.871045 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-1446dcc6-3038-49c6-8a2f-a34364600b9b admin admin] 10.222.0.18 "DELETE /v2.0/subnets/994fc348-a3fc-4763-80fe-d1d9e2dc377a HTTP/1.1" status: 204 len: 168 time: 3.2631929 May 24 11:16:22.879539 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:22.881037 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:22.883663 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-be0eb637-601b-44fb-bcb7-985621098816 req-8016101d-a037-4389-bcc6-2ffce4db0b3a tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 931 time: 0.4828942 May 24 11:16:22.884880 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 51960) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:22.929904 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 7.5746331 May 24 11:16:22.930868 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cc02d756-3752-4aa2-8cb5-4037f6c443b6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:22.936298 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Pushing event updated for resources: {'Port': ['ID=45911d6f-9595-40c5-8b29-75259c824bfe,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:22.944959 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cc02d756-3752-4aa2-8cb5-4037f6c443b6 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:22.950214 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-cc02d756-3752-4aa2-8cb5-4037f6c443b6 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/af222a7b-cca6-4765-946f-530d26ea6c1a HTTP/1.1" status: 204 len: 168 time: 1.8003504 May 24 11:16:22.981684 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.544s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:22.982713 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-af29ed40-83d6-409c-9d17-5d21c28cd8d8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.005s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:23.002875 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cc02d756-3752-4aa2-8cb5-4037f6c443b6 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=af222a7b-cca6-4765-946f-530d26ea6c1a,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:23.046340 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-cc02d756-3752-4aa2-8cb5-4037f6c443b6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:23.102855 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-120fde62-c34f-4db5-a9ec-c28bd1037c95 req-065a8086-0247-4ecd-9299-8207540376d8 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:23.105570 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-120fde62-c34f-4db5-a9ec-c28bd1037c95 req-065a8086-0247-4ecd-9299-8207540376d8 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 931 time: 0.6818347 May 24 11:16:23.107974 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Request body: {'port': {'device_id': '82ca8978-fbb3-4792-8bfd-f8b2e565b925', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:23.120097 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:23.229502 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:23.333053 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Request body: {'port': {'device_id': 'fd3abcad-fe0f-460b-b250-becb3ccb2624', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:23.369964 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-df39f3bc-4453-4db7-9909-b259fb374a69 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:23.488886 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e9c92a19-644e-4ccc-8385-46b18999d6b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:23.632620 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-120fde62-c34f-4db5-a9ec-c28bd1037c95 req-4c536924-e811-414e-b652-bd55fe8c44b2 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.4085984 May 24 11:16:23.633869 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event updated for resources: {'Port': ['ID=777aecd4-bf5c-4efa-a7e0-b3f488ba709c,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:23.642378 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.272s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:23.652373 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:16:23.657404 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:23.661842 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:16:23.723928 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'name': 'tempest-ServerRescueNegativeTestJSON-12783889-router', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233'}], 'enable_snat': True}, 'gw_port_id': 'c9840383-de18-463c-8858-5536335f4466', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:16:04Z', 'updated_at': '2021-05-24T11:16:18Z', 'revision_number': 4, 'project_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'gw_port': {'id': 'c9840383-de18-463c-8858-5536335f4466', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:07:5a:29', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 81, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:06Z', 'updated_at': '2021-05-24T11:16:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '97fb6682-0049-400a-84fe-731434f40cc6', 'name': '', 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'mac_address': 'fa:16:3e:97:4b:6c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'device_owner': 'network:router_interface', 'standard_attr_id': 104, 'fixed_ips': [{'subnet_id': '52267672-fd8c-45bd-89ea-d9d02a7b249a', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:14Z', 'updated_at': '2021-05-24T11:16:18Z', 'revision_number': 2, 'project_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'subnets': [{'id': '52267672-fd8c-45bd-89ea-d9d02a7b249a', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:23.727979 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-250de9b1-b2f1-4647-9ed1-82be32d9501a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:23.729564 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:23.752316 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Checking router: 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:16:23.752678 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Updating router 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 port 97fb6682-0049-400a-84fe-731434f40cc6 binding host dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:16:23.937915 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:23.957533 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3b0bb8ce-54ed-485f-89bf-d2b085d419cb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:23.964870 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:23.996462 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-250de9b1-b2f1-4647-9ed1-82be32d9501a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=df45a7c4-d00d-43d1-a1aa-ebe6a351596e,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:24.000194 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-250de9b1-b2f1-4647-9ed1-82be32d9501a admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:24.003689 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-250de9b1-b2f1-4647-9ed1-82be32d9501a admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/df45a7c4-d00d-43d1-a1aa-ebe6a351596e HTTP/1.1" status: 204 len: 168 time: 2.0156572 May 24 11:16:24.005160 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-250de9b1-b2f1-4647-9ed1-82be32d9501a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:24.104325 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:24.122618 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-be0eb637-601b-44fb-bcb7-985621098816 req-9ab1f3a8-1633-436a-b67f-a8b0dbb74386 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Synchronizing usage tracker for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:24.382833 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:24.414051 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:24.515840 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-be0eb637-601b-44fb-bcb7-985621098816 req-9ab1f3a8-1633-436a-b67f-a8b0dbb74386 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Unset dirty status for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:24.532975 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-be0eb637-601b-44fb-bcb7-985621098816 req-9ab1f3a8-1633-436a-b67f-a8b0dbb74386 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 1.6431623 May 24 11:16:24.922802 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52004) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:25.120563 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9eca99f1-ac8f-4b93-90d8-9208a157db23 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:25.183380 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-9eca99f1-ac8f-4b93-90d8-9208a157db23 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:25.193034 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Notify callbacks [] for port, precommit_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:25.218563 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Transition to ACTIVE for port object 97fb6682-0049-400a-84fe-731434f40cc6 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:16:25.240546 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:25.257744 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-9eca99f1-ac8f-4b93-90d8-9208a157db23 admin admin] Usage tracker for resource:security_group_rule and tenant:245e8947d67c491bb25273d437b63eaa is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:25.272882 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-9eca99f1-ac8f-4b93-90d8-9208a157db23 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:25.280067 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:25.332530 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-9eca99f1-ac8f-4b93-90d8-9208a157db23 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:25.618304 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:25.648519 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > segment before_delete 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:25.649255 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > subnet precommit_delete_associations 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:25.650548 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:25.651060 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > port after_create 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:25.651574 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > port after_delete 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:25.652109 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > port after_update 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:25.652820 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > segment after_delete 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:25.653657 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > subnet after_create 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:25.654527 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:25.656208 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > subnet after_update 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:25.690948 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-571f6efe-4d34-40f0-b8f6-f0fd057f4dde tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:25.692504 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-571f6efe-4d34-40f0-b8f6-f0fd057f4dde tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/networks?id=4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 200 len: 817 time: 1.1215286 May 24 11:16:25.728565 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1969920a-de31-4b65-a8f3-d82eb3bfd2c9 None None] Pushing event updated for resources: {'Port': ['ID=97fb6682-0049-400a-84fe-731434f40cc6,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:25.754134 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-1969920a-de31-4b65-a8f3-d82eb3bfd2c9 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.474s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:25.758200 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-ba122da4-096a-4cf2-865d-b7df6cc14d10 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/quotas/bf5fa11a013d4e1a8b86edc5fe49e97b HTTP/1.1" status: 200 len: 375 time: 0.0406666 May 24 11:16:25.890787 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.db.resource_extend [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] It took 0.13 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 24 11:16:25.916361 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:26.028285 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-61311c56-c069-42e6-8298-74225610a007 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:26.028285 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Attempting to bind port 97fb6682-0049-400a-84fe-731434f40cc6 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:16:26.028285 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Attempting to bind port 97fb6682-0049-400a-84fe-731434f40cc6 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:16:26.028285 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Attempting to bind port 97fb6682-0049-400a-84fe-731434f40cc6 on network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:26.046180 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 16, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:16:26.046925 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Checking segment: {'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:26.047560 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:26.048165 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Bound using segment: {'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:16:26.048948 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Bound port: 97fb6682-0049-400a-84fe-731434f40cc6, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:16:26.049598 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:26.055208 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:26.081942 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:26.096201 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-a210d8cb-779f-4879-b9b9-b55f6aad64ee tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&fields=id HTTP/1.1" status: 200 len: 255 time: 0.3252535 May 24 11:16:26.108604 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6255b6e0-a476-4674-8130-3f75f2e061bd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:26.234163 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] For port 97fb6682-0049-400a-84fe-731434f40cc6, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:16:26.242395 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] For port 97fb6682-0049-400a-84fe-731434f40cc6, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=97fb6682-0049-400a-84fe-731434f40cc6,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:16:26.254455 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:26.287333 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-772f8c5f-f7ca-4c88-b71b-c0a8886a5951 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:26.332040 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > subnet after_create 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:26.332678 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:26.341518 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > subnet before_create 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:26.342175 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Subscribe: > subnet before_update 55550000 {{(pid=91819) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 24 11:16:26.342843 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['c731a0f0-19d8-4a2f-8776-6ddcb26d147c']}} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:26.450773 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Routers {'24e49d59-ce93-4246-ae1a-cf4080a4b8f5'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:16:26.458027 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:16:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['24e49d59-ce93-4246-ae1a-cf4080a4b8f5'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:26.628697 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1969920a-de31-4b65-a8f3-d82eb3bfd2c9 None None] Pushing event updated for resources: {'Port': ['ID=97fb6682-0049-400a-84fe-731434f40cc6,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:26.643670 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-1969920a-de31-4b65-a8f3-d82eb3bfd2c9 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.355s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:26.758225 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 65acda30-2128-434e-a144-f18a69777754 {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:16:26.789587 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:26.843241 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Transition to ACTIVE for port object 65acda30-2128-434e-a144-f18a69777754 will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:16:26.852141 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] Set router 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:16:26.929929 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:26.938490 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'be60d73a-b842-4a9f-ae55-b20e1ec8ab90') {'for_net_delete': True} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:26.947806 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'be60d73a-b842-4a9f-ae55-b20e1ec8ab90') {} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:26.988980 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:27.004059 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-2e8ffd04-becf-41e1-8a31-65bec68dad78 service neutron] 10.222.0.18 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.7994971 May 24 11:16:27.005320 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event updated for resources: {'Port': ['ID=777aecd4-bf5c-4efa-a7e0-b3f488ba709c,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:27.015352 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.907s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:27.241538 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.db.resource_extend [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] It took 0.96 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 24 11:16:27.326741 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9eca99f1-ac8f-4b93-90d8-9208a157db23 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:27.358999 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534853183'] for segment, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:27.376450 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Pushing event updated for resources: {'Port': ['ID=65acda30-2128-434e-a144-f18a69777754,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:27.395503 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.406s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:27.520549 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 45911d6f-9595-40c5-8b29-75259c824bfe {{(pid=91820) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:16:27.535328 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:27.595026 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Transition to ACTIVE for port object 45911d6f-9595-40c5-8b29-75259c824bfe will not be triggered until provisioned by entity L2. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:16:27.608134 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:27.717726 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9eca99f1-ac8f-4b93-90d8-9208a157db23 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:27.719962 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:27.724494 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:27.735830 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:27.736060 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'name': 'tempest-ServerRescueNegativeTestJSON-12783889-router', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233'}], 'enable_snat': True}, 'gw_port_id': 'c9840383-de18-463c-8858-5536335f4466', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:16:04Z', 'updated_at': '2021-05-24T11:16:18Z', 'revision_number': 4, 'project_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'gw_port': {'id': 'c9840383-de18-463c-8858-5536335f4466', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:07:5a:29', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 81, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:06Z', 'updated_at': '2021-05-24T11:16:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '97fb6682-0049-400a-84fe-731434f40cc6', 'name': '', 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'mac_address': 'fa:16:3e:97:4b:6c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'device_owner': 'network:router_interface', 'standard_attr_id': 104, 'fixed_ips': [{'subnet_id': '52267672-fd8c-45bd-89ea-d9d02a7b249a', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:14Z', 'updated_at': '2021-05-24T11:16:26Z', 'revision_number': 4, 'project_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'subnets': [{'id': '52267672-fd8c-45bd-89ea-d9d02a7b249a', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:16:27.740081 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-06c5e20a-30d5-40b9-a337-3ff9ff2d9739 req-f9a6b46f-cabc-4c0d-92a8-293bb99e4aba tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:27.741716 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-06c5e20a-30d5-40b9-a337-3ff9ff2d9739 req-f9a6b46f-cabc-4c0d-92a8-293bb99e4aba tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Synchronizing usage tracker for tenant:e65fade283614b028730cfea0f58bcad on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:27.744336 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b6a8f484-d327-4605-849c-3a3f138ae667 None None] The sync data for ids ['24e49d59-ce93-4246-ae1a-cf4080a4b8f5'] in dv-cinder-784092-9-iscsi is: [{'id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'name': 'tempest-ServerRescueNegativeTestJSON-12783889-router', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233'}], 'enable_snat': True}, 'gw_port_id': 'c9840383-de18-463c-8858-5536335f4466', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:16:04Z', 'updated_at': '2021-05-24T11:16:18Z', 'revision_number': 4, 'project_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'gw_port': {'id': 'c9840383-de18-463c-8858-5536335f4466', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:07:5a:29', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 81, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:06Z', 'updated_at': '2021-05-24T11:16:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '97fb6682-0049-400a-84fe-731434f40cc6', 'name': '', 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'mac_address': 'fa:16:3e:97:4b:6c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'device_owner': 'network:router_interface', 'standard_attr_id': 104, 'fixed_ips': [{'subnet_id': '52267672-fd8c-45bd-89ea-d9d02a7b249a', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:14Z', 'updated_at': '2021-05-24T11:16:26Z', 'revision_number': 4, 'project_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'subnets': [{'id': '52267672-fd8c-45bd-89ea-d9d02a7b249a', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:16:27.847935 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:27.854710 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Releasing vlan 514 on physical network physnet1 to pool {{(pid=91819) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:16:27.856177 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535155490'] for segment, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:27.867852 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 692 time: 11.3014121 May 24 11:16:27.967695 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: from cryptography.utils import int_from_bytes May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: from cryptography.utils import int_from_bytes May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: Traceback (most recent call last): May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: timer() May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: cb(*args, **kw) May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: return func(*args, **kwargs) May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: self._notify() May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: self.callback(batched_events) May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: event.method(event) May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: return f(self, *a, **k) May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._get(url).json() May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: return self.request(url, 'GET', **kwargs) May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:16:28.096468 dv-cinder-784092-9-iscsi neutron-server[91819]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:16:28.114776 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-9eca99f1-ac8f-4b93-90d8-9208a157db23 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=245e8947d67c491bb25273d437b63eaa&name=default HTTP/1.1" status: 200 len: 2979 time: 3.1911550 May 24 11:16:28.126355 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52062) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:28.127903 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9eca99f1-ac8f-4b93-90d8-9208a157db23 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d0d63825-f548-4d2e-801d-ff2bb5f20ca8,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:28.148174 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-9eca99f1-ac8f-4b93-90d8-9208a157db23 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.180s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:28.173559 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.046s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:28.304808 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-06c5e20a-30d5-40b9-a337-3ff9ff2d9739 req-f9a6b46f-cabc-4c0d-92a8-293bb99e4aba tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Unset dirty status for tenant:e65fade283614b028730cfea0f58bcad on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:28.314825 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-06c5e20a-30d5-40b9-a337-3ff9ff2d9739 req-f9a6b46f-cabc-4c0d-92a8-293bb99e4aba tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 931 time: 2.9226518 May 24 11:16:28.335536 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Attempting to bind port 65acda30-2128-434e-a144-f18a69777754 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:16:28.335893 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Attempting to bind port 65acda30-2128-434e-a144-f18a69777754 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': 'b6a611a2-11cb-43b7-8194-22ac1ef4914b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 543, 'network_id': '29272b37-7e2e-4e92-9978-bb32dde1813d'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:16:28.336235 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Attempting to bind port 65acda30-2128-434e-a144-f18a69777754 on network 29272b37-7e2e-4e92-9978-bb32dde1813d {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:28.407993 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Port 65acda30-2128-434e-a144-f18a69777754 on network 29272b37-7e2e-4e92-9978-bb32dde1813d not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:16:28.408414 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Attempting to bind port 65acda30-2128-434e-a144-f18a69777754 on network 29272b37-7e2e-4e92-9978-bb32dde1813d {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:28.434051 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 16, 21), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:16:28.434450 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Checking segment: {'id': 'b6a611a2-11cb-43b7-8194-22ac1ef4914b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 543, 'network_id': '29272b37-7e2e-4e92-9978-bb32dde1813d'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:28.434955 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Bound using segment: {'id': 'b6a611a2-11cb-43b7-8194-22ac1ef4914b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 543, 'network_id': '29272b37-7e2e-4e92-9978-bb32dde1813d'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:16:28.435452 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Bound port: 65acda30-2128-434e-a144-f18a69777754, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'b6a611a2-11cb-43b7-8194-22ac1ef4914b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 543, 'network_id': '29272b37-7e2e-4e92-9978-bb32dde1813d'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:16:28.436176 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:28.470673 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:28.526631 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:28.532609 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-06c5e20a-30d5-40b9-a337-3ff9ff2d9739 req-05843cf4-36f0-43d4-8799-a78402901048 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Synchronizing usage tracker for tenant:e65fade283614b028730cfea0f58bcad on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:28.543169 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:28.608985 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Pushing event deleted for resources: {'Network': ['ID=c731a0f0-19d8-4a2f-8776-6ddcb26d147c,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:28.697739 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.154s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:28.748458 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-356a86fc-78ca-45bc-a5ef-45834c864411 admin admin] 10.222.0.18 "DELETE /v2.0/networks/c731a0f0-19d8-4a2f-8776-6ddcb26d147c HTTP/1.1" status: 204 len: 168 time: 5.8553054 May 24 11:16:28.840224 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-06c5e20a-30d5-40b9-a337-3ff9ff2d9739 req-05843cf4-36f0-43d4-8799-a78402901048 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Unset dirty status for tenant:e65fade283614b028730cfea0f58bcad on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:28.846094 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-06c5e20a-30d5-40b9-a337-3ff9ff2d9739 req-05843cf4-36f0-43d4-8799-a78402901048 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.5300207 May 24 11:16:28.920439 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Pushing event updated for resources: {'Port': ['ID=45911d6f-9595-40c5-8b29-75259c824bfe,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:28.938640 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-60633010-85e0-4844-a796-df2d5f17f7d7 service neutron] 10.222.0.18 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 1.9194098 May 24 11:16:28.942367 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.769s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:28.961295 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] For port 65acda30-2128-434e-a144-f18a69777754, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:16:28.972953 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:28.981531 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] For port 65acda30-2128-434e-a144-f18a69777754, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=65acda30-2128-434e-a144-f18a69777754,segment=NetworkSegment(b6a611a2-11cb-43b7-8194-22ac1ef4914b),segment_id=b6a611a2-11cb-43b7-8194-22ac1ef4914b)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:16:28.986964 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:28.995783 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:29.037778 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 97fb6682-0049-400a-84fe-731434f40cc6 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:29.043919 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Attempting to bind port 45911d6f-9595-40c5-8b29-75259c824bfe on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:16:29.044274 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Attempting to bind port 45911d6f-9595-40c5-8b29-75259c824bfe by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:16:29.044620 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Attempting to bind port 45911d6f-9595-40c5-8b29-75259c824bfe on network bb3e0f6d-2046-4828-8f43-5f99ba5f3add {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:29.068373 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-af9a59fa-1026-40ae-9649-a97fb006becb admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:29.116496 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Port 45911d6f-9595-40c5-8b29-75259c824bfe on network bb3e0f6d-2046-4828-8f43-5f99ba5f3add not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:16:29.117187 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Attempting to bind port 45911d6f-9595-40c5-8b29-75259c824bfe on network bb3e0f6d-2046-4828-8f43-5f99ba5f3add {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:29.159562 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 16, 21), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:16:29.159996 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Checking segment: {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:29.160694 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Bound using segment: {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:16:29.161311 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Bound port: 45911d6f-9595-40c5-8b29-75259c824bfe, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'}}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:16:29.161721 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:29.237953 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:29.239831 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:29.242179 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] 10.222.0.93 "PUT /v2.0/ports/65acda30-2128-434e-a144-f18a69777754 HTTP/1.1" status: 200 len: 1093 time: 6.4602599 May 24 11:16:29.269049 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '82ca8978-fbb3-4792-8bfd-f8b2e565b925', 'tag': '65acda30-2128-434e-a144-f18a69777754'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:16:29.352848 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-af9a59fa-1026-40ae-9649-a97fb006becb admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:29.406932 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 97fb6682-0049-400a-84fe-731434f40cc6 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:29.440505 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 97fb6682-0049-400a-84fe-731434f40cc6 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:29.441059 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:29.448019 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52086) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:29.547307 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-af9a59fa-1026-40ae-9649-a97fb006becb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:29.855471 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Pushing event updated for resources: {'Port': ['ID=65acda30-2128-434e-a144-f18a69777754,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:29.929328 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.913s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:29.978981 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:30.047849 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af9a59fa-1026-40ae-9649-a97fb006becb admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d0d63825-f548-4d2e-801d-ff2bb5f20ca8,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:30.053560 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-1a36ea48-792e-4744-b6dc-b0e39be89be4 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?name=tempest-securitygroup--618302121&fields=id&tenant_id=b303f83a84794e5ba6a1bab9c76b429c HTTP/1.1" status: 200 len: 267 time: 0.3040900 May 24 11:16:30.061576 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-af9a59fa-1026-40ae-9649-a97fb006becb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:30.079915 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-af9a59fa-1026-40ae-9649-a97fb006becb admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:30.085796 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-af9a59fa-1026-40ae-9649-a97fb006becb admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/d0d63825-f548-4d2e-801d-ff2bb5f20ca8 HTTP/1.1" status: 204 len: 168 time: 1.9349122 May 24 11:16:30.089698 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-9be392f4-3cbc-4be8-84fd-51fe265a6c8f admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=de2d8ae871924896af66ae6c97b6ee1b&name=default HTTP/1.1" status: 200 len: 2975 time: 0.6378875 May 24 11:16:30.097852 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-55a485e4-9ca8-4770-8542-2f2ea741b223 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:30.101500 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-55a485e4-9ca8-4770-8542-2f2ea741b223 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=e65fade283614b028730cfea0f58bcad&device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 1096 time: 0.7682247 May 24 11:16:30.102034 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 52116) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:30.129813 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:30.201721 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 97fb6682-0049-400a-84fe-731434f40cc6, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=97fb6682-0049-400a-84fe-731434f40cc6,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:30.209200 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:30.212765 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:30.300017 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] For port 45911d6f-9595-40c5-8b29-75259c824bfe, host hv-cinder-78409, cleared binding levels {{(pid=91820) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:16:30.476147 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=97fb6682-0049-400a-84fe-731434f40cc6,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:30.477371 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] For port 45911d6f-9595-40c5-8b29-75259c824bfe, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=45911d6f-9595-40c5-8b29-75259c824bfe,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91820) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:16:30.480206 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.267s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:30.483351 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-bd674b10-009a-4abb-80d3-90a5bb488093 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=65acda30-2128-434e-a144-f18a69777754 HTTP/1.1" status: 200 len: 217 time: 0.3664310 May 24 11:16:30.492054 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f4fe6070-e281-42a5-8321-4e50ba185651 req-c680059f-4aa6-4ba4-95d7-ea98fba1f3b5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:30.496928 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:30.500712 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f4fe6070-e281-42a5-8321-4e50ba185651 req-c680059f-4aa6-4ba4-95d7-ea98fba1f3b5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 943 time: 0.9856141 May 24 11:16:30.508221 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:30.605325 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:30.606292 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:30.609444 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] 10.222.0.93 "PUT /v2.0/ports/45911d6f-9595-40c5-8b29-75259c824bfe HTTP/1.1" status: 200 len: 1093 time: 7.6334639 May 24 11:16:30.613004 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'fd3abcad-fe0f-460b-b250-becb3ccb2624', 'tag': '45911d6f-9595-40c5-8b29-75259c824bfe'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:16:30.655423 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Pushing event updated for resources: {'Port': ['ID=45911d6f-9595-40c5-8b29-75259c824bfe,revision_number=3']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:30.665790 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-9f22d5fe-745a-4add-bd97-5a2e9bc9b177 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:30.667961 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-9f22d5fe-745a-4add-bd97-5a2e9bc9b177 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/networks?id=1d92641a-d8f9-4ed6-9abd-f464bb26811d HTTP/1.1" status: 200 len: 811 time: 0.5771105 May 24 11:16:30.668943 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.161s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:30.684350 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-5b98f789-99cc-4aca-97af-5c2258fb30ac tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/quotas/b303f83a84794e5ba6a1bab9c76b429c HTTP/1.1" status: 200 len: 375 time: 0.0103402 May 24 11:16:30.774647 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-53b9b4ef-127a-4276-aa4a-6a101b874264 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:30.813406 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f4fe6070-e281-42a5-8321-4e50ba185651 req-a812ada0-1d2b-4cac-b40f-e960457816e0 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3116429 May 24 11:16:30.821810 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 97fb6682-0049-400a-84fe-731434f40cc6 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:30.914351 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-53b9b4ef-127a-4276-aa4a-6a101b874264 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:30.940281 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-0e4201cb-a749-4e92-9595-13c817313c71 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Synchronizing usage tracker for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:30.946773 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-f21b37b6-595c-420f-88c4-7a2a9f380a23 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:30.948998 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-f21b37b6-595c-420f-88c4-7a2a9f380a23 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 1096 time: 0.2543924 May 24 11:16:30.994431 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-c4cb8190-44ad-4068-bfba-1ce1dd43fc30 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:30.997051 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-c4cb8190-44ad-4068-bfba-1ce1dd43fc30 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=20bb60b1-11d7-4cba-b636-3bcdb98876e9 HTTP/1.1" status: 200 len: 852 time: 0.3608186 May 24 11:16:30.998496 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-53b9b4ef-127a-4276-aa4a-6a101b874264 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:31.014352 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 97fb6682-0049-400a-84fe-731434f40cc6 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:31.017662 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:31.060894 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-6acd465d-205b-45ca-ad4d-65f83b1d6c0f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:31.089611 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-0e4201cb-a749-4e92-9595-13c817313c71 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Unset dirty status for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:31.093013 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-0e4201cb-a749-4e92-9595-13c817313c71 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=b303f83a84794e5ba6a1bab9c76b429c&fields=id HTTP/1.1" status: 200 len: 348 time: 0.3999875 May 24 11:16:31.094072 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-2b10c322-c33a-4014-bb58-20f4c1a8957f service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=45911d6f-9595-40c5-8b29-75259c824bfe HTTP/1.1" status: 200 len: 217 time: 0.1299438 May 24 11:16:31.155223 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-0578b61a-e92b-4719-87a6-e5416cea9b45 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=29272b37-7e2e-4e92-9978-bb32dde1813d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1494908 May 24 11:16:31.166582 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-f0b072bd-d369-4075-b9f5-07820990fd3f req-1ceae753-d343-4b97-85f6-76a687ac1794 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Synchronizing usage tracker for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:31.301268 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0d1b6984-1f99-489d-9be7-91ac5752d8b0 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:31.303586 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-0d1b6984-1f99-489d-9be7-91ac5752d8b0 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=1107b450-6c27-4f13-8b6e-abdbbbb1ea0f HTTP/1.1" status: 200 len: 854 time: 0.1990891 May 24 11:16:31.372558 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-eae8bfcb-6b8e-40c4-8964-309aba34b20c service neutron] 10.222.0.93 "GET /v2.0/networks/29272b37-7e2e-4e92-9978-bb32dde1813d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2081327 May 24 11:16:31.373950 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-53b9b4ef-127a-4276-aa4a-6a101b874264 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=73805a20-c072-4ab0-bffb-f5df0cc148d2,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:31.376945 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-53b9b4ef-127a-4276-aa4a-6a101b874264 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.316s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:31.379854 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-53b9b4ef-127a-4276-aa4a-6a101b874264 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:31.385045 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-53b9b4ef-127a-4276-aa4a-6a101b874264 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/73805a20-c072-4ab0-bffb-f5df0cc148d2 HTTP/1.1" status: 204 len: 168 time: 1.2729082 May 24 11:16:31.393309 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-93352f99-7d72-4519-9dcd-117e1b70f03b service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0813663 May 24 11:16:31.547425 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1edd6f29-d45d-4d0b-afb8-61dca0e690ce req-c65b1e22-01f3-468c-ac76-e8841df9ea57 service neutron] 10.222.0.93 "GET /v2.0/networks/29272b37-7e2e-4e92-9978-bb32dde1813d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1633081 May 24 11:16:31.548720 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-f0b072bd-d369-4075-b9f5-07820990fd3f req-1ceae753-d343-4b97-85f6-76a687ac1794 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Unset dirty status for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:31.571908 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f0b072bd-d369-4075-b9f5-07820990fd3f req-1ceae753-d343-4b97-85f6-76a687ac1794 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 210 time: 0.5717046 May 24 11:16:31.630380 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-64b35d66-7395-48da-8382-b124db6a3191 service neutron] 10.222.0.18 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4731739 May 24 11:16:32.103621 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-ffc1546c-18fa-4dec-b4be-516144d38544 service neutron] 10.222.0.18 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4710038 May 24 11:16:32.195655 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-c046d2df-2184-4939-b8e7-f73c982affc1 service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.7933962 May 24 11:16:32.247865 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-a7304e83-0838-436d-947c-c9ea1232f24b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:32.248874 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-a7304e83-0838-436d-947c-c9ea1232f24b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Synchronizing usage tracker for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:network {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:32.271017 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-96653d86-ed2d-4edf-8e72-81433353b961 req-7eca45d1-8276-4996-9e63-fdda0321994f tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:32.273486 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-96653d86-ed2d-4edf-8e72-81433353b961 req-7eca45d1-8276-4996-9e63-fdda0321994f tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 943 time: 0.3014810 May 24 11:16:32.274009 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 52190) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:32.469566 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b9a2e12-c874-450c-8753-ba047e94e75d req-f9e1adde-4f68-4a5b-b501-d71e34875b0f service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2656605 May 24 11:16:32.498221 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-a7304e83-0838-436d-947c-c9ea1232f24b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Unset dirty status for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:network {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:32.500054 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-a7304e83-0838-436d-947c-c9ea1232f24b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.93 "GET /v2.0/networks?id=4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 200 len: 817 time: 0.8938146 May 24 11:16:32.533464 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Request body: {'port': {'device_id': 'd9e36d1d-343c-4b97-b226-9b58722aebb2', 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714', 'admin_state_up': True, 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:32.633991 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52204) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:32.634186 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-bd54349c-fb22-432f-97c5-d767d9e57b4b admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=cb09a77682654da58923a77805e9b68a&name=default HTTP/1.1" status: 200 len: 2975 time: 0.3580394 May 24 11:16:32.650698 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-96653d86-ed2d-4edf-8e72-81433353b961 req-12e121db-32bc-4238-b99c-58cb28ba85b3 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2153153 May 24 11:16:32.807091 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:32.826954 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attempting to reserve 1 items for resource port. Total usage: 1; quota limit: 500; headroom:499 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:32.847695 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Made reservation on behalf of bf5fa11a013d4e1a8b86edc5fe49e97b for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:32.861026 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:32.909834 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-148ca4e6-0138-4789-af4b-167d14d2a562 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:33.131632 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-148ca4e6-0138-4789-af4b-167d14d2a562 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:33.257695 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'fd3abcad-fe0f-460b-b250-becb3ccb2624', 'tag': '45911d6f-9595-40c5-8b29-75259c824bfe', 'status': 'completed', 'code': 200} May 24 11:16:33.321297 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '82ca8978-fbb3-4792-8bfd-f8b2e565b925', 'tag': '65acda30-2128-434e-a144-f18a69777754', 'status': 'completed', 'code': 200} May 24 11:16:33.321897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.93', 65220) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:33.382125 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-148ca4e6-0138-4789-af4b-167d14d2a562 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:33.404775 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-46e0c73f-e3d3-409b-af58-86fb297a947e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:33.475272 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-148ca4e6-0138-4789-af4b-167d14d2a562 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b284fa57-77ac-481f-a5db-b2621c00144b,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:33.496624 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-148ca4e6-0138-4789-af4b-167d14d2a562 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:33.501304 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-148ca4e6-0138-4789-af4b-167d14d2a562 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/b284fa57-77ac-481f-a5db-b2621c00144b HTTP/1.1" status: 204 len: 168 time: 0.8647678 May 24 11:16:33.505195 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-148ca4e6-0138-4789-af4b-167d14d2a562 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:33.511536 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-06632fdf-de09-4d33-b865-73e25a52c85f req-34682392-7e39-47b2-a388-14ca9af929c4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:33.515371 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-06632fdf-de09-4d33-b865-73e25a52c85f req-34682392-7e39-47b2-a388-14ca9af929c4 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 1096 time: 0.2467387 May 24 11:16:33.677332 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Allocated IP 172.20.0.10 (4fec6b68-64f1-4553-8ca5-f86c63ac9714/52267672-fd8c-45bd-89ea-d9d02a7b249a/02210ade-8c60-41b6-aa82-a104d4fc0447) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:16:33.966231 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-06632fdf-de09-4d33-b865-73e25a52c85f req-2dad222d-eb5b-49e2-9505-f93f62fcaafb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:33.967755 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-06632fdf-de09-4d33-b865-73e25a52c85f req-2dad222d-eb5b-49e2-9505-f93f62fcaafb service neutron] 10.222.0.93 "GET /v2.0/networks?id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add HTTP/1.1" status: 200 len: 915 time: 0.4453888 May 24 11:16:34.007567 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 65acda30-2128-434e-a144-f18a69777754 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:16:34.049358 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8218fc8b-9a50-42ca-b288-d04c266de1dc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:34.051428 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8218fc8b-9a50-42ca-b288-d04c266de1dc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/networks?id=4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 200 len: 817 time: 0.4709501 May 24 11:16:34.062315 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-06632fdf-de09-4d33-b865-73e25a52c85f req-06d503c7-488d-456e-8543-b7da7ce9ba56 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=45911d6f-9595-40c5-8b29-75259c824bfe HTTP/1.1" status: 200 len: 217 time: 0.0814292 May 24 11:16:34.119096 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ae06923-6713-4aaf-8deb-cb28c954b825 req-530bb198-5e7d-4567-a2bf-6d1c6872ca63 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:34.120852 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4ae06923-6713-4aaf-8deb-cb28c954b825 req-530bb198-5e7d-4567-a2bf-6d1c6872ca63 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=e65fade283614b028730cfea0f58bcad&device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 1096 time: 0.7976494 May 24 11:16:34.145344 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-20038432-6148-4691-80dc-401425a22206 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/quotas/bf5fa11a013d4e1a8b86edc5fe49e97b HTTP/1.1" status: 200 len: 375 time: 0.0854287 May 24 11:16:34.163956 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-def749b8-bbb9-4787-a96a-4ee35e0f449d req-52b946f4-0573-4767-a4fb-d82f3f413d10 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:34.182532 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-def749b8-bbb9-4787-a96a-4ee35e0f449d req-52b946f4-0573-4767-a4fb-d82f3f413d10 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 943 time: 0.5939889 May 24 11:16:34.292852 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 65acda30-2128-434e-a144-f18a69777754, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=65acda30-2128-434e-a144-f18a69777754,segment=NetworkSegment(b6a611a2-11cb-43b7-8194-22ac1ef4914b),segment_id=b6a611a2-11cb-43b7-8194-22ac1ef4914b)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:34.318270 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '65acda30-2128-434e-a144-f18a69777754', 'network_id': '29272b37-7e2e-4e92-9978-bb32dde1813d', 'port_id': '65acda30-2128-434e-a144-f18a69777754', 'mac_address': 'fa:16:3e:02:5f:1a', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 543, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '20bb60b1-11d7-4cba-b636-3bcdb98876e9', 'ip_address': '172.20.0.5'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:16:34.350073 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-06632fdf-de09-4d33-b865-73e25a52c85f req-67bd94ad-6ca4-4b02-8cab-a0d239ca3b95 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:34.351581 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-06632fdf-de09-4d33-b865-73e25a52c85f req-67bd94ad-6ca4-4b02-8cab-a0d239ca3b95 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=1107b450-6c27-4f13-8b6e-abdbbbb1ea0f HTTP/1.1" status: 200 len: 854 time: 0.2746811 May 24 11:16:34.439320 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-4626f586-1918-4ba0-854e-12fc2732cb64 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&fields=id HTTP/1.1" status: 200 len: 255 time: 0.2770209 May 24 11:16:34.475131 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-def749b8-bbb9-4787-a96a-4ee35e0f449d req-649eaec5-dc8b-4482-997f-fa1f088619cd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2914958 May 24 11:16:34.538182 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:34.540890 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:34.603600 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 65acda30-2128-434e-a144-f18a69777754 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:16:34.621539 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ae06923-6713-4aaf-8deb-cb28c954b825 req-c4bd2398-ec8a-4b25-837f-3c2b7a8147bd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:34.623709 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4ae06923-6713-4aaf-8deb-cb28c954b825 req-c4bd2398-ec8a-4b25-837f-3c2b7a8147bd service neutron] 10.222.0.93 "GET /v2.0/networks?id=29272b37-7e2e-4e92-9978-bb32dde1813d HTTP/1.1" status: 200 len: 913 time: 0.4386413 May 24 11:16:34.643180 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 65acda30-2128-434e-a144-f18a69777754, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=65acda30-2128-434e-a144-f18a69777754,segment=NetworkSegment(b6a611a2-11cb-43b7-8194-22ac1ef4914b),segment_id=b6a611a2-11cb-43b7-8194-22ac1ef4914b)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:34.650267 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:34.653472 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:34.655938 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:34.693070 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:34.767463 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=65acda30-2128-434e-a144-f18a69777754,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:34.772794 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.119s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:34.799099 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-ca0001f5-b97d-4d61-94d3-625147c8cacb service neutron] 10.222.0.18 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3490744 May 24 11:16:34.844922 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6923f431-2d25-43e4-afc1-f7e03250bd63 req-220627a7-8329-4e48-9ec4-123116b33411 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 210 time: 0.0488775 May 24 11:16:34.877663 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:16:34.891789 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:34.900637 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:16:34.987934 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Pushing event updated for resources: {'Port': ['ID=02210ade-8c60-41b6-aa82-a104d4fc0447,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:35.003030 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.310s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:35.014779 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:35.015593 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:35.027275 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-e8ed1b81-3f45-4142-b93b-57eb9b509186 service neutron] 10.222.0.18 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2209363 May 24 11:16:35.153818 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 2.6274717 May 24 11:16:35.182525 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9552b42b-9d94-4945-8d04-55cfd50146ca req-3fe474a6-f0d1-4474-a8b5-11381ac6219f tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:35.184275 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9552b42b-9d94-4945-8d04-55cfd50146ca req-3fe474a6-f0d1-4474-a8b5-11381ac6219f tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 944 time: 0.1580820 May 24 11:16:35.295276 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9552b42b-9d94-4945-8d04-55cfd50146ca req-239934ff-a82a-4ecf-ab21-b9353ff52814 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0985293 May 24 11:16:35.308921 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b654ecc4-8046-4122-aa41-0489d71e1497 req-70832115-6c6a-4d4d-ae0e-b73d840a85b8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0639706 May 24 11:16:35.820446 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e7d2a7e2-b1c2-4384-a272-7c06a1d0886a req-0e17d4a7-fab4-48fe-8f38-bd78e93c93d7 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:35.829727 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e7d2a7e2-b1c2-4384-a272-7c06a1d0886a req-0e17d4a7-fab4-48fe-8f38-bd78e93c93d7 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 943 time: 0.1510878 May 24 11:16:35.907121 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e7d2a7e2-b1c2-4384-a272-7c06a1d0886a req-1ecc920e-89ba-4064-af60-6eb906b9ce22 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0764258 May 24 11:16:36.087849 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8b691040-c32b-4f02-b62c-e3b2239e080b req-53262253-7a93-455e-889a-7af1dead2f9c tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 210 time: 0.0624278 May 24 11:16:36.556543 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1344f9f9-f23f-48c0-a899-3fea6b5e12bf req-b55e7908-680f-4b5f-a05d-eb3a32817a29 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0624774 May 24 11:16:36.620302 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-56c6fae5-7620-41f2-bff3-afefa2571d9e req-eee049bc-dd96-4955-a1b0-fe2176e72332 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:36.621719 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-56c6fae5-7620-41f2-bff3-afefa2571d9e req-eee049bc-dd96-4955-a1b0-fe2176e72332 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 944 time: 0.1692851 May 24 11:16:36.683884 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-56c6fae5-7620-41f2-bff3-afefa2571d9e req-21927a62-98f1-44c5-8347-6348fffb8d53 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0552201 May 24 11:16:37.210299 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e828ce1e-cbb4-4f43-9a14-414a890b6d27 req-5221055e-8e86-49ad-9078-a0b8057c1b23 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:37.213418 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-e828ce1e-cbb4-4f43-9a14-414a890b6d27 req-5221055e-8e86-49ad-9078-a0b8057c1b23 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Synchronizing usage tracker for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:37.246377 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-e828ce1e-cbb4-4f43-9a14-414a890b6d27 req-5221055e-8e86-49ad-9078-a0b8057c1b23 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Unset dirty status for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:37.247744 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e828ce1e-cbb4-4f43-9a14-414a890b6d27 req-5221055e-8e86-49ad-9078-a0b8057c1b23 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 943 time: 0.1854408 May 24 11:16:37.367918 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e828ce1e-cbb4-4f43-9a14-414a890b6d27 req-ab6d9aeb-bb84-49b3-bd68-61f0c5f4277d tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1098778 May 24 11:16:37.382598 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e6f6b243-4d9f-4b64-bb5c-7ee48dbde816 req-e9f6e2ac-8280-4990-a958-6e63cfe3ba6c tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 210 time: 0.0747917 May 24 11:16:37.384031 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 45911d6f-9595-40c5-8b29-75259c824bfe details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:16:37.385213 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.93', 65223) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:37.511440 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-06632fdf-de09-4d33-b865-73e25a52c85f req-78eac546-7e86-42f8-9a3e-bae7544bbec3 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1117177 May 24 11:16:37.709799 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4ae06923-6713-4aaf-8deb-cb28c954b825 req-dc1057a3-a3dc-4ba7-b891-52cbf4e02042 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=65acda30-2128-434e-a144-f18a69777754 HTTP/1.1" status: 200 len: 217 time: 0.3150749 May 24 11:16:37.725760 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-06632fdf-de09-4d33-b865-73e25a52c85f req-b505daf9-9bc7-4ad6-bad7-7b2457e8a0ef service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2048340 May 24 11:16:37.740214 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Request body: {'port': {'device_id': 'd9e36d1d-343c-4b97-b226-9b58722aebb2', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:37.800148 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 45911d6f-9595-40c5-8b29-75259c824bfe, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=45911d6f-9595-40c5-8b29-75259c824bfe,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:37.876134 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '45911d6f-9595-40c5-8b29-75259c824bfe', 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add', 'port_id': '45911d6f-9595-40c5-8b29-75259c824bfe', 'mac_address': 'fa:16:3e:ca:a8:43', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 524, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '1107b450-6c27-4f13-8b6e-abdbbbb1ea0f', 'ip_address': '172.20.0.8'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:16:37.885872 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['65acda30-2128-434e-a144-f18a69777754'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:16:37.999123 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52484) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:38.128799 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-14df91ca-08de-4be9-a8f2-16728dad9841 req-ab5e9dc1-e15a-47e3-9d49-01f3b0f0e672 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.1303771 May 24 11:16:38.305029 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-06632fdf-de09-4d33-b865-73e25a52c85f req-f5647a7a-cab4-4e32-8103-b6b4d44703c9 service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.5720091 May 24 11:16:38.309211 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:38.363919 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 45911d6f-9595-40c5-8b29-75259c824bfe {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:16:38.444432 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e6b58333-8b71-4626-b4de-3c16765c2cc5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:38.472656 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 45911d6f-9595-40c5-8b29-75259c824bfe, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=45911d6f-9595-40c5-8b29-75259c824bfe,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:38.479831 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-e6b58333-8b71-4626-b4de-3c16765c2cc5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:38.488988 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-e6b58333-8b71-4626-b4de-3c16765c2cc5 admin admin] Usage tracker for resource:security_group_rule and tenant:7691d492dc9a43dfa66fddae213fa160 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:38.493793 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-e6b58333-8b71-4626-b4de-3c16765c2cc5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:38.520505 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:38.526467 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-e6b58333-8b71-4626-b4de-3c16765c2cc5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:38.535583 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:38.805046 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['45911d6f-9595-40c5-8b29-75259c824bfe'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:16:38.818265 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-bc18e324-2e65-46c6-9deb-387f699ef5bc tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:38.819322 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-bc18e324-2e65-46c6-9deb-387f699ef5bc tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Synchronizing usage tracker for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:network {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:38.834921 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=45911d6f-9595-40c5-8b29-75259c824bfe,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:38.844902 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.309s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:38.873352 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ae06923-6713-4aaf-8deb-cb28c954b825 req-aba287d1-c8a6-4355-aa0a-d3a5b725f308 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:38.880787 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4ae06923-6713-4aaf-8deb-cb28c954b825 req-aba287d1-c8a6-4355-aa0a-d3a5b725f308 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=20bb60b1-11d7-4cba-b636-3bcdb98876e9 HTTP/1.1" status: 200 len: 852 time: 1.0201359 May 24 11:16:38.918048 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:39.241909 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e862aecb-d4d5-46d9-91ca-1fd948894b35 req-d1a036c7-c0f8-40ab-8786-d3b742d1af0b tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:39.250600 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e862aecb-d4d5-46d9-91ca-1fd948894b35 req-d1a036c7-c0f8-40ab-8786-d3b742d1af0b tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 944 time: 0.3572218 May 24 11:16:39.263461 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-9cb9d1a5-1a0c-4c8f-8d6a-daeed34b1314 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:39.265036 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-9cb9d1a5-1a0c-4c8f-8d6a-daeed34b1314 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.93 "GET /v2.0/networks?id=4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 200 len: 817 time: 0.4065273 May 24 11:16:39.304864 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3939838b-eca7-4df7-8caf-c2c56fee8341 req-539d8100-a732-4148-92f0-22e089e21675 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:39.315996 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3939838b-eca7-4df7-8caf-c2c56fee8341 req-539d8100-a732-4148-92f0-22e089e21675 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 944 time: 0.8684709 May 24 11:16:39.317353 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Request body: {'port': {'device_id': 'ef0f174b-4c60-4917-8c38-60dd5c2c0add', 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714', 'admin_state_up': True, 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:39.359330 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4ae06923-6713-4aaf-8deb-cb28c954b825 req-e53e52c8-3a58-4796-8c51-d28945e61514 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=29272b37-7e2e-4e92-9978-bb32dde1813d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.3618300 May 24 11:16:39.399830 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 65acda30-2128-434e-a144-f18a69777754 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:39.509720 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e6b58333-8b71-4626-b4de-3c16765c2cc5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:39.537958 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e862aecb-d4d5-46d9-91ca-1fd948894b35 req-eea0cd8d-9d7d-4537-8de9-412815897fc6 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2860119 May 24 11:16:39.565421 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e6b58333-8b71-4626-b4de-3c16765c2cc5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:39.575245 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4ae06923-6713-4aaf-8deb-cb28c954b825 req-a70cb9be-9e77-4572-8a07-9a3ffcc51d40 service neutron] 10.222.0.93 "GET /v2.0/networks/29272b37-7e2e-4e92-9978-bb32dde1813d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1958020 May 24 11:16:39.605332 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 65acda30-2128-434e-a144-f18a69777754 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:39.613990 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 65acda30-2128-434e-a144-f18a69777754 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:39.616980 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:39.630949 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:39.653688 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:39.700951 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Usage tracker for resource:port and tenant:bf5fa11a013d4e1a8b86edc5fe49e97b is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:39.740706 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Quota usage for port was recalculated. Used quota:2. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:39.761962 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:39.769325 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-e6b58333-8b71-4626-b4de-3c16765c2cc5 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=7691d492dc9a43dfa66fddae213fa160&name=default HTTP/1.1" status: 200 len: 2979 time: 1.7646060 May 24 11:16:39.778265 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52538) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:39.807189 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-bc18e324-2e65-46c6-9deb-387f699ef5bc tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Unset dirty status for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:network {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:39.811364 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e6b58333-8b71-4626-b4de-3c16765c2cc5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=75fd0862-5d7f-4e5a-95e1-43a0d16b7bcb,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:39.826032 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-bc18e324-2e65-46c6-9deb-387f699ef5bc tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.93 "GET /v2.0/networks?id=1d92641a-d8f9-4ed6-9abd-f464bb26811d HTTP/1.1" status: 200 len: 811 time: 2.4125450 May 24 11:16:39.840624 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3939838b-eca7-4df7-8caf-c2c56fee8341 req-2dadeff7-7586-427a-b21b-4e26f10d23ce tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.5232444 May 24 11:16:39.935751 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-fa394bf4-0939-4571-925c-6c137e2fd0ef req-82f3e3e2-2de7-4a4e-97e1-c10c487d7b4a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 210 time: 0.1409295 May 24 11:16:39.946323 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e6b58333-8b71-4626-b4de-3c16765c2cc5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.315s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:39.956038 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Made reservation on behalf of bf5fa11a013d4e1a8b86edc5fe49e97b for: {'port': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:39.979635 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:39.983233 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:40.036640 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '82ca8978-fbb3-4792-8bfd-f8b2e565b925', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '65acda30-2128-434e-a144-f18a69777754'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:16:40.116877 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 65acda30-2128-434e-a144-f18a69777754, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=65acda30-2128-434e-a144-f18a69777754,segment=NetworkSegment(b6a611a2-11cb-43b7-8194-22ac1ef4914b),segment_id=b6a611a2-11cb-43b7-8194-22ac1ef4914b)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:40.125394 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:40.129040 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:40.221133 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=65acda30-2128-434e-a144-f18a69777754,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:40.234057 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.105s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:40.249628 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-47c4ddc8-9771-439b-84eb-572930717615 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:40.313442 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-47c4ddc8-9771-439b-84eb-572930717615 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:40.431915 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-8b42f377-7907-4883-9f1d-77bc91190f8d tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Synchronizing usage tracker for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:40.508100 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-47c4ddc8-9771-439b-84eb-572930717615 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:40.528120 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:40.553184 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-ebb00954-64c0-4a04-a4c0-b9e59ee83387 req-9cf1bb59-e801-4bee-b825-34a481643307 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Synchronizing usage tracker for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:40.589230 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-47c4ddc8-9771-439b-84eb-572930717615 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=75fd0862-5d7f-4e5a-95e1-43a0d16b7bcb,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:40.594348 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-47c4ddc8-9771-439b-84eb-572930717615 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.066s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:40.607789 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-47c4ddc8-9771-439b-84eb-572930717615 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:40.609117 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-47c4ddc8-9771-439b-84eb-572930717615 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/75fd0862-5d7f-4e5a-95e1-43a0d16b7bcb HTTP/1.1" status: 204 len: 168 time: 0.8216088 May 24 11:16:40.637697 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-8b42f377-7907-4883-9f1d-77bc91190f8d tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Unset dirty status for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:40.639825 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-8b42f377-7907-4883-9f1d-77bc91190f8d tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.93 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=b303f83a84794e5ba6a1bab9c76b429c HTTP/1.1" status: 200 len: 379 time: 0.8124430 May 24 11:16:40.681718 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Request body: {'port': {'device_id': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'admin_state_up': True, 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'security_groups': ['20a7f42b-2874-40d1-94c8-66d210ae913e']}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:40.778261 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '82ca8978-fbb3-4792-8bfd-f8b2e565b925', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '65acda30-2128-434e-a144-f18a69777754', 'code': 200} May 24 11:16:40.820619 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4ae06923-6713-4aaf-8deb-cb28c954b825 req-74f5089d-b30b-47fc-b8ed-09485f0e4c39 service neutron] 10.222.0.93 "GET /v2.0/networks/29272b37-7e2e-4e92-9978-bb32dde1813d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 1.2336080 May 24 11:16:40.827002 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-ebb00954-64c0-4a04-a4c0-b9e59ee83387 req-9cf1bb59-e801-4bee-b825-34a481643307 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Unset dirty status for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:40.837795 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ebb00954-64c0-4a04-a4c0-b9e59ee83387 req-9cf1bb59-e801-4bee-b825-34a481643307 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.4318106 May 24 11:16:41.133566 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c9335ca5-bfea-4147-ab9d-c639e12ce39b req-0ad006de-3285-464f-9456-fd8d5b1784dd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:41.141613 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c9335ca5-bfea-4147-ab9d-c639e12ce39b req-0ad006de-3285-464f-9456-fd8d5b1784dd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 944 time: 0.1594946 May 24 11:16:41.204573 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c9335ca5-bfea-4147-ab9d-c639e12ce39b req-8f58a52d-7d30-4c04-9ecc-56fde77f8132 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0616000 May 24 11:16:41.408966 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 45911d6f-9595-40c5-8b29-75259c824bfe up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:41.462228 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52612) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:41.478649 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 02210ade-8c60-41b6-aa82-a104d4fc0447 {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:16:41.497049 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:41.517727 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Transition to ACTIVE for port object 02210ade-8c60-41b6-aa82-a104d4fc0447 will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:16:41.542192 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5e399cbc-8c58-4186-b2ab-cee64a70e6b7 req-ed33df12-c1c4-4d7c-b61d-90c0dd2d9c5c tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 210 time: 0.1558301 May 24 11:16:41.549413 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:41.568353 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:41.597151 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:41.597758 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 45911d6f-9595-40c5-8b29-75259c824bfe completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:41.606293 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 45911d6f-9595-40c5-8b29-75259c824bfe triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:41.606947 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:41.637614 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:41.653486 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ae904b1c-0eab-49e5-a0cc-3bdd0910104e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:41.691301 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-ae904b1c-0eab-49e5-a0cc-3bdd0910104e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:41.700246 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-ae904b1c-0eab-49e5-a0cc-3bdd0910104e admin admin] Usage tracker for resource:security_group_rule and tenant:ea552c44ce7b45cc83c985c80f1a1112 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:41.705050 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-ae904b1c-0eab-49e5-a0cc-3bdd0910104e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:41.716133 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-ae904b1c-0eab-49e5-a0cc-3bdd0910104e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:41.768262 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Made reservation on behalf of b303f83a84794e5ba6a1bab9c76b429c for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:41.772475 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:41.788016 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52622) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:41.833040 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Allocated IP 172.20.0.11 (4fec6b68-64f1-4553-8ca5-f86c63ac9714/52267672-fd8c-45bd-89ea-d9d02a7b249a/692ab816-0fdb-448b-9b27-c44c0a457f4d) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:16:41.841437 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:41.946015 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Pushing event updated for resources: {'Port': ['ID=02210ade-8c60-41b6-aa82-a104d4fc0447,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:41.951548 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 45911d6f-9595-40c5-8b29-75259c824bfe, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=45911d6f-9595-40c5-8b29-75259c824bfe,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:41.959772 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:41.962653 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:41.969927 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.373s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:41.984894 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ae904b1c-0eab-49e5-a0cc-3bdd0910104e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.064649 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93d22ff3-87fc-4de3-8c69-c0c3158d6143 req-0f1f489a-721b-41a7-a888-7677c1e3ba36 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:42.066535 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-93d22ff3-87fc-4de3-8c69-c0c3158d6143 req-0f1f489a-721b-41a7-a888-7677c1e3ba36 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 1098 time: 0.8087001 May 24 11:16:42.067352 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-4638d1b2-69d8-4662-929f-0c56286d9729 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.076750 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ae904b1c-0eab-49e5-a0cc-3bdd0910104e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.092158 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=45911d6f-9595-40c5-8b29-75259c824bfe,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:42.095519 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:42.096136 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.133s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:42.124020 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dbc87c42-3b6f-401b-bd69-a047c93c715c req-e4461fb3-110a-47db-8993-021c60751503 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:42.125584 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dbc87c42-3b6f-401b-bd69-a047c93c715c req-e4461fb3-110a-47db-8993-021c60751503 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 945 time: 0.8762741 May 24 11:16:42.135244 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-93d22ff3-87fc-4de3-8c69-c0c3158d6143 req-afb46b31-38b8-4d62-9b88-2d0cb3479b15 service neutron] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.156916 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-4638d1b2-69d8-4662-929f-0c56286d9729 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:42.183843 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-4638d1b2-69d8-4662-929f-0c56286d9729 admin admin] Usage tracker for resource:security_group_rule and tenant:6ade2747f8604c69905711615d40e57c is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:42.190867 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-93d22ff3-87fc-4de3-8c69-c0c3158d6143 req-afb46b31-38b8-4d62-9b88-2d0cb3479b15 service neutron] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:42.194241 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-4638d1b2-69d8-4662-929f-0c56286d9729 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:42.204596 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-9b0e815a-931a-4a9e-b191-89d1cdf245d4 req-0e9661c1-62b5-4e1d-b3f8-4bf18f0714b4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Synchronizing usage tracker for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:42.226260 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-4638d1b2-69d8-4662-929f-0c56286d9729 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:42.234932 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-93d22ff3-87fc-4de3-8c69-c0c3158d6143 req-afb46b31-38b8-4d62-9b88-2d0cb3479b15 service neutron] Usage tracker for resource:security_group_rule and tenant:d712e044a57f4a8ea6ee57fae590ba23 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:42.238967 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ae904b1c-0eab-49e5-a0cc-3bdd0910104e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7874cd6d-4698-49eb-91c6-885aeaf8a662,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:42.242062 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ae904b1c-0eab-49e5-a0cc-3bdd0910104e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.147s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:42.243099 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-93d22ff3-87fc-4de3-8c69-c0c3158d6143 req-afb46b31-38b8-4d62-9b88-2d0cb3479b15 service neutron] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:42.248126 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Attempting to bind port 02210ade-8c60-41b6-aa82-a104d4fc0447 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:16:42.248420 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Attempting to bind port 02210ade-8c60-41b6-aa82-a104d4fc0447 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:16:42.253082 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Attempting to bind port 02210ade-8c60-41b6-aa82-a104d4fc0447 on network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:42.253792 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-ae904b1c-0eab-49e5-a0cc-3bdd0910104e admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=ea552c44ce7b45cc83c985c80f1a1112&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7712224 May 24 11:16:42.264690 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52632) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:42.269662 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-93d22ff3-87fc-4de3-8c69-c0c3158d6143 req-afb46b31-38b8-4d62-9b88-2d0cb3479b15 service neutron] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:42.297991 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Port 02210ade-8c60-41b6-aa82-a104d4fc0447 on network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:16:42.298323 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Attempting to bind port 02210ade-8c60-41b6-aa82-a104d4fc0447 on network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:42.317319 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dbc87c42-3b6f-401b-bd69-a047c93c715c req-1240dd7c-07cb-4cb4-8c50-73834830dd10 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1692836 May 24 11:16:42.321242 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-9b0e815a-931a-4a9e-b191-89d1cdf245d4 req-0e9661c1-62b5-4e1d-b3f8-4bf18f0714b4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Unset dirty status for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:42.322408 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9b0e815a-931a-4a9e-b191-89d1cdf245d4 req-0e9661c1-62b5-4e1d-b3f8-4bf18f0714b4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.2278478 May 24 11:16:42.323259 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 16, 21), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:16:42.324457 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Checking segment: {'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:42.324858 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Bound using segment: {'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:16:42.325803 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Bound port: 02210ade-8c60-41b6-aa82-a104d4fc0447, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:16:42.325803 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.350143 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.353503 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Allocated IP 172.20.0.12 (1d92641a-d8f9-4ed6-9abd-f464bb26811d/2fc4be2e-875c-4d10-b857-a48f90c53cf2/eab44150-44f7-4194-9b76-436d421794e8) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:16:42.455959 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-824e4b4a-2983-48dd-9261-de7b280e41a9 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.496486 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.545847 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-8d3d00c9-0b7f-4e88-8e8d-e158052ffffb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:42.666539 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Pushing event updated for resources: {'Port': ['ID=692ab816-0fdb-448b-9b27-c44c0a457f4d,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:42.679597 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-4638d1b2-69d8-4662-929f-0c56286d9729 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.682172 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-824e4b4a-2983-48dd-9261-de7b280e41a9 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.690601 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.145s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:42.709125 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-4638d1b2-69d8-4662-929f-0c56286d9729 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.723632 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:16:42.779093 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'fd3abcad-fe0f-460b-b250-becb3ccb2624', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '45911d6f-9595-40c5-8b29-75259c824bfe'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:16:42.791112 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-148ca4e6-0138-4789-af4b-167d14d2a562 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:42.792340 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.799056 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:42.803445 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9e61825b-d3be-4ed2-a99f-5d8253424c39 req-cb5022b5-2534-4d88-a063-f890e73830a1 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 210 time: 0.0584006 May 24 11:16:42.810863 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:16:42.850663 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Transition to ACTIVE for port object eab44150-44f7-4194-9b76-436d421794e8 will not be triggered until provisioned by entity DHCP. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:16:42.869886 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] For port 02210ade-8c60-41b6-aa82-a104d4fc0447, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:16:42.908322 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] For port 02210ade-8c60-41b6-aa82-a104d4fc0447, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=02210ade-8c60-41b6-aa82-a104d4fc0447,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:16:42.928078 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4638d1b2-69d8-4662-929f-0c56286d9729 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1e9d1f0d-8cb2-4361-ad0b-a2e1b89d372d,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:42.932802 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.935585 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-93d22ff3-87fc-4de3-8c69-c0c3158d6143 req-afb46b31-38b8-4d62-9b88-2d0cb3479b15 service neutron] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.947379 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-4638d1b2-69d8-4662-929f-0c56286d9729 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=6ade2747f8604c69905711615d40e57c&name=default HTTP/1.1" status: 200 len: 2979 time: 1.1572418 May 24 11:16:42.952393 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-4638d1b2-69d8-4662-929f-0c56286d9729 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.160s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:42.952393 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52652) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:42.961695 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2a0dd00d-b311-44f9-8fe4-297c7978909c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:42.974910 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:42.975696 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:42.979763 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'fd3abcad-fe0f-460b-b250-becb3ccb2624', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '45911d6f-9595-40c5-8b29-75259c824bfe', 'code': 200} May 24 11:16:43.027011 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-93d22ff3-87fc-4de3-8c69-c0c3158d6143 req-afb46b31-38b8-4d62-9b88-2d0cb3479b15 service neutron] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:43.086053 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eaef3f10-9ec2-40ae-a849-08dbb40e740a req-3de52ab3-8a56-4315-a998-991c9ef4a4a5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:43.087553 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-eaef3f10-9ec2-40ae-a849-08dbb40e740a req-3de52ab3-8a56-4315-a998-991c9ef4a4a5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 945 time: 0.6068377 May 24 11:16:43.107532 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:43.108677 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:43.110389 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] 10.222.0.93 "PUT /v2.0/ports/02210ade-8c60-41b6-aa82-a104d4fc0447 HTTP/1.1" status: 200 len: 1094 time: 5.7060826 May 24 11:16:43.111516 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'd9e36d1d-343c-4b97-b226-9b58722aebb2', 'tag': '02210ade-8c60-41b6-aa82-a104d4fc0447'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:16:43.140108 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:43.147676 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-93d22ff3-87fc-4de3-8c69-c0c3158d6143 req-afb46b31-38b8-4d62-9b88-2d0cb3479b15 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 1.0713773 May 24 11:16:43.188806 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Pushing event updated for resources: {'Port': ['ID=02210ade-8c60-41b6-aa82-a104d4fc0447,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:43.198249 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Network 1d92641a-d8f9-4ed6-9abd-f464bb26811d is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:16:43.198913 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.237s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:43.199509 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.123s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:43.213668 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:43.222646 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:16:43.269523 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'd9e36d1d-343c-4b97-b226-9b58722aebb2', 'tag': '02210ade-8c60-41b6-aa82-a104d4fc0447', 'status': 'completed', 'code': 200} May 24 11:16:43.288248 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-824e4b4a-2983-48dd-9261-de7b280e41a9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:43.292545 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-eaef3f10-9ec2-40ae-a849-08dbb40e740a req-d5fcdc3b-f9c3-4932-adc7-c1e1c3692a15 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1951807 May 24 11:16:43.332224 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-afb46b31-38b8-4d62-9b88-2d0cb3479b15 service neutron] Pushing event updated for resources: {'SecurityGroup': ['ID=6ffc4202-6107-45cd-9238-b4ff90966621,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:43.351251 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-afb46b31-38b8-4d62-9b88-2d0cb3479b15 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.151s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:43.351846 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.185s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:43.368586 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-af9a59fa-1026-40ae-9649-a97fb006becb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:43.391441 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-824e4b4a-2983-48dd-9261-de7b280e41a9 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:43.396194 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-824e4b4a-2983-48dd-9261-de7b280e41a9 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/7874cd6d-4698-49eb-91c6-885aeaf8a662 HTTP/1.1" status: 204 len: 168 time: 1.1270258 May 24 11:16:43.407530 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-824e4b4a-2983-48dd-9261-de7b280e41a9 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7874cd6d-4698-49eb-91c6-885aeaf8a662,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:43.413667 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 4.1000824 May 24 11:16:43.417131 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-824e4b4a-2983-48dd-9261-de7b280e41a9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.068s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:43.443427 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Request body: {'port': {'device_id': 'ef0f174b-4c60-4917-8c38-60dd5c2c0add', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:43.620239 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-056bb1bb-72d0-4bbd-a8dd-690da7a255a7 None None] Provisioning for port eab44150-44f7-4194-9b76-436d421794e8 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:43.656018 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-056bb1bb-72d0-4bbd-a8dd-690da7a255a7 None None] Provisioning complete for port eab44150-44f7-4194-9b76-436d421794e8 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:43.657631 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-056bb1bb-72d0-4bbd-a8dd-690da7a255a7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:43.777272 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f7f60d14-f1ba-4258-80cd-730c4922371f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:43.881744 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:43.882786 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:43.957932 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-056bb1bb-72d0-4bbd-a8dd-690da7a255a7 None None] Port eab44150-44f7-4194-9b76-436d421794e8 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:16:44.037412 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 3.3843598 May 24 11:16:44.044028 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event updated for resources: {'Port': ['ID=eab44150-44f7-4194-9b76-436d421794e8,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:44.065123 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f7f60d14-f1ba-4258-80cd-730c4922371f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:44.066967 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Request body: {'port': {'device_id': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:44.119598 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:44.164633 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.812s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:44.274228 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c8f9817e-106b-4992-93fc-22bd958365aa req-d1d62c4d-078b-4c16-9be9-2e07bc2fb005 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:44.276937 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c8f9817e-106b-4992-93fc-22bd958365aa req-d1d62c4d-078b-4c16-9be9-2e07bc2fb005 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 1098 time: 0.8366473 May 24 11:16:44.331652 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3134dd33-7bef-4261-be37-d5e47a0e0abd req-9aa04704-bebf-4d21-a371-dea52442b02f tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:44.334270 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3134dd33-7bef-4261-be37-d5e47a0e0abd req-9aa04704-bebf-4d21-a371-dea52442b02f tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 945 time: 0.5266201 May 24 11:16:44.399587 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-11e301df-a999-4952-8dd3-2467abf687df service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:44.401627 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-11e301df-a999-4952-8dd3-2467abf687df service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 1097 time: 0.9736352 May 24 11:16:44.439495 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6db9fe10-b073-4df8-ac8b-a7f7ab3f0470 req-cfd4ec86-5491-4d37-b4d3-7bebc93a27c7 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:44.442282 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6db9fe10-b073-4df8-ac8b-a7f7ab3f0470 req-cfd4ec86-5491-4d37-b4d3-7bebc93a27c7 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 932 time: 0.6731942 May 24 11:16:44.451529 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f7f60d14-f1ba-4258-80cd-730c4922371f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:44.490245 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:44.552881 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f7f60d14-f1ba-4258-80cd-730c4922371f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1e9d1f0d-8cb2-4361-ad0b-a2e1b89d372d,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:44.587616 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f7f60d14-f1ba-4258-80cd-730c4922371f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:44.589004 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f7f60d14-f1ba-4258-80cd-730c4922371f admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:44.590579 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-f7f60d14-f1ba-4258-80cd-730c4922371f admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/1e9d1f0d-8cb2-4361-ad0b-a2e1b89d372d HTTP/1.1" status: 204 len: 168 time: 1.6373222 May 24 11:16:44.602381 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-7c779365-4bc2-498e-8f2d-60e07a3f2943 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=02210ade-8c60-41b6-aa82-a104d4fc0447 HTTP/1.1" status: 200 len: 217 time: 0.1819844 May 24 11:16:44.662888 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-6db9fe10-b073-4df8-ac8b-a7f7ab3f0470 req-fd96a162-365e-435d-bf50-21de29360181 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Synchronizing usage tracker for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:44.665584 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:44.693984 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3134dd33-7bef-4261-be37-d5e47a0e0abd req-d5a78b3a-b826-400d-861d-4b7e7502b2c2 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2802763 May 24 11:16:44.924057 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-c8f9817e-106b-4992-93fc-22bd958365aa req-7e0d6c9b-d24c-4159-af93-40bd332e64a5 service neutron] Synchronizing usage tracker for tenant:d712e044a57f4a8ea6ee57fae590ba23 on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:45.010570 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-6db9fe10-b073-4df8-ac8b-a7f7ab3f0470 req-fd96a162-365e-435d-bf50-21de29360181 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Unset dirty status for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:45.012575 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6db9fe10-b073-4df8-ac8b-a7f7ab3f0470 req-fd96a162-365e-435d-bf50-21de29360181 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.4234016 May 24 11:16:45.022363 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-b6bfe0e9-1779-43d9-bdcc-af6b755f47ed service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:45.024673 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-b6bfe0e9-1779-43d9-bdcc-af6b755f47ed service neutron] 10.222.0.93 "GET /v2.0/subnets?id=52267672-fd8c-45bd-89ea-d9d02a7b249a HTTP/1.1" status: 200 len: 857 time: 0.3684418 May 24 11:16:45.071987 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-c8f9817e-106b-4992-93fc-22bd958365aa req-7e0d6c9b-d24c-4159-af93-40bd332e64a5 service neutron] Unset dirty status for tenant:d712e044a57f4a8ea6ee57fae590ba23 on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:45.073948 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c8f9817e-106b-4992-93fc-22bd958365aa req-7e0d6c9b-d24c-4159-af93-40bd332e64a5 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.7873328 May 24 11:16:45.218055 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 692ab816-0fdb-448b-9b27-c44c0a457f4d {{(pid=91820) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:16:45.243871 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:45.289007 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-2445a81d-409e-4d44-b4f7-5e8ca44d87f1 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=4fec6b68-64f1-4553-8ca5-f86c63ac9714&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.2179642 May 24 11:16:45.323138 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Transition to ACTIVE for port object 692ab816-0fdb-448b-9b27-c44c0a457f4d will not be triggered until provisioned by entity L2. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:16:45.335435 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:45.346196 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:45.414798 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eeccec1c-f2c4-4e45-97c0-4521b9e3fd4a req-3b10fbe1-e7ea-41db-ac3a-778cbe98f836 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:45.416862 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-eeccec1c-f2c4-4e45-97c0-4521b9e3fd4a req-3b10fbe1-e7ea-41db-ac3a-778cbe98f836 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 932 time: 0.4836218 May 24 11:16:45.644108 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 52740) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:45.674121 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-765036c9-9934-4590-be2b-11ca75016812 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:45.677600 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Pushing event updated for resources: {'Port': ['ID=692ab816-0fdb-448b-9b27-c44c0a457f4d,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:45.697421 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.350s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:45.705303 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-765036c9-9934-4590-be2b-11ca75016812 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:45.713401 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-765036c9-9934-4590-be2b-11ca75016812 admin admin] Usage tracker for resource:security_group_rule and tenant:f4cd106b0d9a46bc9ef1119500e201d0 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:45.727062 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-765036c9-9934-4590-be2b-11ca75016812 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:45.760366 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-765036c9-9934-4590-be2b-11ca75016812 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:45.825877 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-d551f208-a249-4c46-8577-12445cdbb868 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4153397 May 24 11:16:45.842393 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d600ea4c-9088-426c-9aa8-3ffc2b3f9f86 req-8f7d553c-e320-4960-8eb4-adf4d7bbf40b tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:45.854184 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d600ea4c-9088-426c-9aa8-3ffc2b3f9f86 req-8f7d553c-e320-4960-8eb4-adf4d7bbf40b tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 945 time: 0.5400357 May 24 11:16:46.015658 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-eeccec1c-f2c4-4e45-97c0-4521b9e3fd4a req-88e6c194-38c5-4a4e-98d5-3599c199592a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.5806880 May 24 11:16:46.122537 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Attempting to bind port 692ab816-0fdb-448b-9b27-c44c0a457f4d on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:16:46.123019 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Attempting to bind port 692ab816-0fdb-448b-9b27-c44c0a457f4d by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:16:46.123740 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Attempting to bind port 692ab816-0fdb-448b-9b27-c44c0a457f4d on network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:46.136387 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Port 692ab816-0fdb-448b-9b27-c44c0a457f4d on network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:16:46.136387 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Attempting to bind port 692ab816-0fdb-448b-9b27-c44c0a457f4d on network 4fec6b68-64f1-4553-8ca5-f86c63ac9714 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:46.163927 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 16, 21), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:16:46.164634 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Checking segment: {'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:46.165410 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Bound using segment: {'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:16:46.166226 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Bound port: 692ab816-0fdb-448b-9b27-c44c0a457f4d, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '6bc6a763-352e-4ddf-bea5-3721ee687b85', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 519, 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714'}}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:16:46.170078 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:46.178785 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d600ea4c-9088-426c-9aa8-3ffc2b3f9f86 req-6d79b1bc-a382-490b-b07b-7d73c621bf08 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3226471 May 24 11:16:46.394578 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] For port 692ab816-0fdb-448b-9b27-c44c0a457f4d, host hv-cinder-78409, cleared binding levels {{(pid=91820) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:16:46.400394 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] For port 692ab816-0fdb-448b-9b27-c44c0a457f4d, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=692ab816-0fdb-448b-9b27-c44c0a457f4d,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91820) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:16:46.417384 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-add70568-c8dd-4a52-9651-4c72f68b2cf4 req-e89e37ad-128c-4b98-a318-40e33fabc922 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:46.425276 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-add70568-c8dd-4a52-9651-4c72f68b2cf4 req-e89e37ad-128c-4b98-a318-40e33fabc922 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 945 time: 0.2047417 May 24 11:16:46.428770 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:46.464660 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:46.467010 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cf61ffe4-1d65-414d-9580-8cbaf3b2a978 req-ff61de16-7d86-43d3-b3d8-9e898828cc22 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.5627940 May 24 11:16:46.487905 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id eab44150-44f7-4194-9b76-436d421794e8 {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:16:46.495815 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-765036c9-9934-4590-be2b-11ca75016812 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:46.504375 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:46.510158 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-765036c9-9934-4590-be2b-11ca75016812 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:46.559526 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Pushing event updated for resources: {'Port': ['ID=692ab816-0fdb-448b-9b27-c44c0a457f4d,revision_number=3']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:46.566436 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:46.567054 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:46.569010 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] 10.222.0.93 "PUT /v2.0/ports/692ab816-0fdb-448b-9b27-c44c0a457f4d HTTP/1.1" status: 200 len: 1094 time: 3.1428750 May 24 11:16:46.569826 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.105s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:46.581672 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'ef0f174b-4c60-4917-8c38-60dd5c2c0add', 'tag': '692ab816-0fdb-448b-9b27-c44c0a457f4d'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:16:46.613711 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-27711c9c-51e6-4897-9df0-0b4d8eb5b9a1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:46.623901 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-add70568-c8dd-4a52-9651-4c72f68b2cf4 req-98188e96-7383-4cda-83dc-85b504b94209 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1958828 May 24 11:16:46.670537 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Transition to ACTIVE for port object eab44150-44f7-4194-9b76-436d421794e8 will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:16:46.708367 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-765036c9-9934-4590-be2b-11ca75016812 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ea271af5-c88a-41a9-a907-007f823a7b08,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:46.718889 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:46.723672 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-765036c9-9934-4590-be2b-11ca75016812 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=f4cd106b0d9a46bc9ef1119500e201d0&name=default HTTP/1.1" status: 200 len: 2979 time: 1.0733581 May 24 11:16:46.725219 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-765036c9-9934-4590-be2b-11ca75016812 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:46.733181 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 52768) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:46.735338 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'ef0f174b-4c60-4917-8c38-60dd5c2c0add', 'tag': '692ab816-0fdb-448b-9b27-c44c0a457f4d', 'status': 'completed', 'code': 200} May 24 11:16:46.760320 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:46.775224 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-69ba276e-6818-4a85-b41a-873dfb0364a1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:46.779710 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-69ba276e-6818-4a85-b41a-873dfb0364a1 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 1097 time: 0.1545141 May 24 11:16:46.887872 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-ea828a23-e5b8-4496-a44a-524d71f1f8e4 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=692ab816-0fdb-448b-9b27-c44c0a457f4d HTTP/1.1" status: 200 len: 217 time: 0.1023629 May 24 11:16:47.037666 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1a58155-4721-4524-84cb-8d156c13c510 req-ba02dc3c-0653-4c33-9a37-2bea22526462 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:47.039982 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f1a58155-4721-4524-84cb-8d156c13c510 req-ba02dc3c-0653-4c33-9a37-2bea22526462 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 944 time: 0.6353483 May 24 11:16:47.092969 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-80ca2311-309c-47d5-ba63-3f565cff73a5 req-fa061074-b425-40da-88f0-a1d21a74c315 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:47.094530 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-80ca2311-309c-47d5-ba63-3f565cff73a5 req-fa061074-b425-40da-88f0-a1d21a74c315 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 1097 time: 0.5612750 May 24 11:16:47.095226 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-4edf789c-90c5-4e95-9bb5-825ccc8ac9e8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:47.097223 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-4edf789c-90c5-4e95-9bb5-825ccc8ac9e8 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=52267672-fd8c-45bd-89ea-d9d02a7b249a HTTP/1.1" status: 200 len: 857 time: 0.2025666 May 24 11:16:47.168133 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Pushing event updated for resources: {'Port': ['ID=eab44150-44f7-4194-9b76-436d421794e8,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:47.170286 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-952e0347-aad0-4efd-9d1f-e38e3ce730c6 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=4fec6b68-64f1-4553-8ca5-f86c63ac9714&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0630686 May 24 11:16:47.261288 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.501s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:47.326084 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b2e8ebe9-db23-4e9a-ad31-8f371b2ab49a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:47.339374 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f1a58155-4721-4524-84cb-8d156c13c510 req-2eb42979-6d93-4263-8be2-30fdfe60457a tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2547536 May 24 11:16:47.426959 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b2e8ebe9-db23-4e9a-ad31-8f371b2ab49a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:47.535517 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-211e1ae1-8990-47b2-9bfc-bfb690e9b472 req-3b4d42b6-7e80-4278-8a57-cdc432fc88d1 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:47.544816 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-211e1ae1-8990-47b2-9bfc-bfb690e9b472 req-3b4d42b6-7e80-4278-8a57-cdc432fc88d1 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 944 time: 0.3351488 May 24 11:16:47.617207 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-e7b9e68b-d673-43d9-be11-8aa7e794d9d9 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4390337 May 24 11:16:47.627415 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-211e1ae1-8990-47b2-9bfc-bfb690e9b472 req-23aeb6c8-2b0e-48f6-b72f-cf922f1e4271 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0814764 May 24 11:16:47.667187 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Attempting to bind port eab44150-44f7-4194-9b76-436d421794e8 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:16:47.667568 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Attempting to bind port eab44150-44f7-4194-9b76-436d421794e8 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:16:47.667898 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Attempting to bind port eab44150-44f7-4194-9b76-436d421794e8 on network 1d92641a-d8f9-4ed6-9abd-f464bb26811d {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:47.672152 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b2e8ebe9-db23-4e9a-ad31-8f371b2ab49a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:47.737664 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-765036c9-9934-4590-be2b-11ca75016812 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:47.759087 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Port eab44150-44f7-4194-9b76-436d421794e8 on network 1d92641a-d8f9-4ed6-9abd-f464bb26811d not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:16:47.759399 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Attempting to bind port eab44150-44f7-4194-9b76-436d421794e8 on network 1d92641a-d8f9-4ed6-9abd-f464bb26811d {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:16:47.771615 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b2e8ebe9-db23-4e9a-ad31-8f371b2ab49a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ea271af5-c88a-41a9-a907-007f823a7b08,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:47.791648 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b2e8ebe9-db23-4e9a-ad31-8f371b2ab49a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:47.793307 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b2e8ebe9-db23-4e9a-ad31-8f371b2ab49a admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:47.820765 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-b2e8ebe9-db23-4e9a-ad31-8f371b2ab49a admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/ea271af5-c88a-41a9-a907-007f823a7b08 HTTP/1.1" status: 204 len: 168 time: 1.0488410 May 24 11:16:47.830338 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-80ca2311-309c-47d5-ba63-3f565cff73a5 req-406f6a91-160c-440c-9959-3f54c1543ea4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:47.835372 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-80ca2311-309c-47d5-ba63-3f565cff73a5 req-406f6a91-160c-440c-9959-3f54c1543ea4 service neutron] 10.222.0.93 "GET /v2.0/networks?id=4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 200 len: 918 time: 0.7271638 May 24 11:16:47.838729 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 16, 21), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:16:47.839546 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Checking segment: {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:47.840391 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Bound using segment: {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:16:47.841449 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Bound port: eab44150-44f7-4194-9b76-436d421794e8, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:16:47.842204 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:47.879782 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a1338139-6fe1-43ff-a8cc-fbbb647ff689 req-2c77b90c-9c6b-4019-9566-708c13d5c076 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2487655 May 24 11:16:47.949534 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-80ca2311-309c-47d5-ba63-3f565cff73a5 req-ecddd294-cee0-455d-9db4-6e18deabf8f3 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=02210ade-8c60-41b6-aa82-a104d4fc0447 HTTP/1.1" status: 200 len: 217 time: 0.0878310 May 24 11:16:48.079773 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-31a180c3-fb32-49b0-a2eb-8b305189a45e req-4327c538-3a71-44b4-977e-9ccba8c85508 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:48.080989 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-31a180c3-fb32-49b0-a2eb-8b305189a45e req-4327c538-3a71-44b4-977e-9ccba8c85508 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 1097 time: 0.1468611 May 24 11:16:48.397591 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5de92df-6e7c-415c-b6fa-09697bbaea52 req-3f7c294d-0068-4e83-9e0c-cfbd06520f6f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:48.408533 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b5de92df-6e7c-415c-b6fa-09697bbaea52 req-3f7c294d-0068-4e83-9e0c-cfbd06520f6f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 945 time: 0.7735736 May 24 11:16:48.415090 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-80ca2311-309c-47d5-ba63-3f565cff73a5 req-48851ed9-ed21-4e97-803c-0a388db93c7e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:48.422127 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-80ca2311-309c-47d5-ba63-3f565cff73a5 req-48851ed9-ed21-4e97-803c-0a388db93c7e service neutron] 10.222.0.93 "GET /v2.0/subnets?id=52267672-fd8c-45bd-89ea-d9d02a7b249a HTTP/1.1" status: 200 len: 857 time: 0.3106472 May 24 11:16:48.428488 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] For port eab44150-44f7-4194-9b76-436d421794e8, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:16:48.432286 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] For port eab44150-44f7-4194-9b76-436d421794e8, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=eab44150-44f7-4194-9b76-436d421794e8,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:16:48.492188 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-31a180c3-fb32-49b0-a2eb-8b305189a45e req-0a53b2a1-d312-4294-8f1c-c5e7fedea0ef service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:48.494266 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-31a180c3-fb32-49b0-a2eb-8b305189a45e req-0a53b2a1-d312-4294-8f1c-c5e7fedea0ef service neutron] 10.222.0.93 "GET /v2.0/networks?id=4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 200 len: 918 time: 0.4047372 May 24 11:16:48.510471 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-80ca2311-309c-47d5-ba63-3f565cff73a5 req-21ed905c-1eaa-4e07-bb74-f2c0d6c0d6ab service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=4fec6b68-64f1-4553-8ca5-f86c63ac9714&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0570624 May 24 11:16:48.511546 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:48.529618 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:48.777201 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-31a180c3-fb32-49b0-a2eb-8b305189a45e req-dfdaf920-e423-4696-bf91-fb34b34a2464 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=692ab816-0fdb-448b-9b27-c44c0a457f4d HTTP/1.1" status: 200 len: 217 time: 0.2666538 May 24 11:16:48.795161 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:48.795779 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:48.798305 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] 10.222.0.93 "PUT /v2.0/ports/eab44150-44f7-4194-9b76-436d421794e8 HTTP/1.1" status: 200 len: 1094 time: 4.7446978 May 24 11:16:48.813066 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b5de92df-6e7c-415c-b6fa-09697bbaea52 req-e4df4270-e6a2-4b79-9240-1726b4696fa4 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.4032309 May 24 11:16:48.813769 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'tag': 'eab44150-44f7-4194-9b76-436d421794e8'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:16:48.970532 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-31a180c3-fb32-49b0-a2eb-8b305189a45e req-87631432-977c-4149-88cf-48074ca2d523 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:48.972340 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-31a180c3-fb32-49b0-a2eb-8b305189a45e req-87631432-977c-4149-88cf-48074ca2d523 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=52267672-fd8c-45bd-89ea-d9d02a7b249a HTTP/1.1" status: 200 len: 857 time: 0.1716490 May 24 11:16:49.072926 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-31a180c3-fb32-49b0-a2eb-8b305189a45e req-6af36981-eef5-4274-9b6b-045517f32061 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=4fec6b68-64f1-4553-8ca5-f86c63ac9714&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0810466 May 24 11:16:49.101790 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Pushing event updated for resources: {'Port': ['ID=eab44150-44f7-4194-9b76-436d421794e8,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:49.115160 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.585s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:49.117967 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-6162f06a-8478-42a5-9d9a-f2b892b5390a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:49.119382 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-6162f06a-8478-42a5-9d9a-f2b892b5390a service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=b303f83a84794e5ba6a1bab9c76b429c&device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 1097 time: 0.2297609 May 24 11:16:49.209760 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-fbc4a2bf-6bd9-483b-ab3c-c6bb17ca9453 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=eab44150-44f7-4194-9b76-436d421794e8 HTTP/1.1" status: 200 len: 217 time: 0.0795050 May 24 11:16:49.217887 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-80ca2311-309c-47d5-ba63-3f565cff73a5 req-6cbb3ad2-eb70-4752-83ec-9221de0a4df2 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.6910827 May 24 11:16:49.349648 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-252f16cb-bcbd-405c-a117-ed56b400e143 req-d125cba1-b783-4b8c-bbd4-b237831485c0 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:49.352868 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-252f16cb-bcbd-405c-a117-ed56b400e143 req-d125cba1-b783-4b8c-bbd4-b237831485c0 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 944 time: 0.6974349 May 24 11:16:49.437502 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 692ab816-0fdb-448b-9b27-c44c0a457f4d details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:16:49.524346 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-252f16cb-bcbd-405c-a117-ed56b400e143 req-4f9370d5-78d6-4641-9db7-57b98a95f3f1 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1613462 May 24 11:16:49.550650 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-31a180c3-fb32-49b0-a2eb-8b305189a45e req-f91ec340-6f20-476c-ae73-0e91ca00d2ea service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4649212 May 24 11:16:49.554863 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-d2475455-ae88-417c-9a5e-5f76e50e8b3e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:49.556922 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-d2475455-ae88-417c-9a5e-5f76e50e8b3e service neutron] 10.222.0.93 "GET /v2.0/subnets?id=2fc4be2e-875c-4d10-b857-a48f90c53cf2 HTTP/1.1" status: 200 len: 850 time: 0.3281407 May 24 11:16:49.843815 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-80ca2311-309c-47d5-ba63-3f565cff73a5 req-e49405eb-795e-4d10-b881-c8eaefd7683a service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.6162620 May 24 11:16:49.848783 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 692ab816-0fdb-448b-9b27-c44c0a457f4d, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=692ab816-0fdb-448b-9b27-c44c0a457f4d,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:49.883442 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '692ab816-0fdb-448b-9b27-c44c0a457f4d', 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714', 'port_id': '692ab816-0fdb-448b-9b27-c44c0a457f4d', 'mac_address': 'fa:16:3e:59:10:b5', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 519, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '52267672-fd8c-45bd-89ea-d9d02a7b249a', 'ip_address': '172.20.0.11'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:16:49.946703 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-0da8db42-1bc3-4e95-9a9e-56e028d8242e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:49.951086 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-31a180c3-fb32-49b0-a2eb-8b305189a45e req-1da5b598-e0f1-4951-b952-32c023ada672 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3939090 May 24 11:16:49.952686 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-0da8db42-1bc3-4e95-9a9e-56e028d8242e service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=1d92641a-d8f9-4ed6-9abd-f464bb26811d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.3868229 May 24 11:16:49.986818 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:50.016522 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 692ab816-0fdb-448b-9b27-c44c0a457f4d {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:16:50.059967 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 692ab816-0fdb-448b-9b27-c44c0a457f4d, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=692ab816-0fdb-448b-9b27-c44c0a457f4d,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:50.067994 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:50.068767 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 02210ade-8c60-41b6-aa82-a104d4fc0447 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:16:50.082411 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:50.140839 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-952d3686-65d4-4b9b-97fa-e85cd6ebebb6 req-c73a607c-7156-49d8-b159-8372c24d4f61 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:50.161039 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-952d3686-65d4-4b9b-97fa-e85cd6ebebb6 req-c73a607c-7156-49d8-b159-8372c24d4f61 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 944 time: 0.2079740 May 24 11:16:50.256347 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'tag': 'eab44150-44f7-4194-9b76-436d421794e8', 'status': 'completed', 'code': 200} May 24 11:16:50.279822 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-952d3686-65d4-4b9b-97fa-e85cd6ebebb6 req-9b339cf5-a22b-4eb9-858c-b59c34ea3479 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1171763 May 24 11:16:50.310419 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=692ab816-0fdb-448b-9b27-c44c0a457f4d,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:50.315913 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.233s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:50.349823 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 02210ade-8c60-41b6-aa82-a104d4fc0447, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=02210ade-8c60-41b6-aa82-a104d4fc0447,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:50.361340 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-babf9453-8bec-48b5-a604-17269816668a service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3901386 May 24 11:16:50.376123 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '02210ade-8c60-41b6-aa82-a104d4fc0447', 'network_id': '4fec6b68-64f1-4553-8ca5-f86c63ac9714', 'port_id': '02210ade-8c60-41b6-aa82-a104d4fc0447', 'mac_address': 'fa:16:3e:bf:a7:f6', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 519, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '52267672-fd8c-45bd-89ea-d9d02a7b249a', 'ip_address': '172.20.0.10'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:16:50.462996 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:50.490759 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 02210ade-8c60-41b6-aa82-a104d4fc0447 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:16:50.523564 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 02210ade-8c60-41b6-aa82-a104d4fc0447, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=02210ade-8c60-41b6-aa82-a104d4fc0447,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:50.531102 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:50.534211 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:50.575376 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a40dded8-5fcc-4a72-a0f5-dcfed6901d96 req-eb0ef606-983a-4c87-bccf-e9bad023b522 service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2113867 May 24 11:16:50.652163 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=02210ade-8c60-41b6-aa82-a104d4fc0447,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:50.673513 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.139s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:50.834266 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88ed182b-1711-410e-b672-ddf6d7d5b11c req-6cb71110-2859-45e9-8c68-e71f1b055d53 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:50.835462 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['692ab816-0fdb-448b-9b27-c44c0a457f4d'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:16:50.835772 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-88ed182b-1711-410e-b672-ddf6d7d5b11c req-6cb71110-2859-45e9-8c68-e71f1b055d53 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=b303f83a84794e5ba6a1bab9c76b429c&device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 1097 time: 0.1812844 May 24 11:16:50.848888 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c47961c-e380-47fb-bdec-622f0eeb37dd req-d11876ad-5489-4587-bf12-5944e316d041 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:50.850578 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4c47961c-e380-47fb-bdec-622f0eeb37dd req-d11876ad-5489-4587-bf12-5944e316d041 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 945 time: 0.1226151 May 24 11:16:50.890088 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['02210ade-8c60-41b6-aa82-a104d4fc0447'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:16:51.090729 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4c47961c-e380-47fb-bdec-622f0eeb37dd req-74cfd19e-3779-4f8e-b2fc-5d03b20135d5 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2124457 May 24 11:16:51.190767 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88ed182b-1711-410e-b672-ddf6d7d5b11c req-60b85b54-a9b5-4d62-83f8-31d8c0fda296 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:51.192114 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-88ed182b-1711-410e-b672-ddf6d7d5b11c req-60b85b54-a9b5-4d62-83f8-31d8c0fda296 service neutron] 10.222.0.93 "GET /v2.0/networks?id=1d92641a-d8f9-4ed6-9abd-f464bb26811d HTTP/1.1" status: 200 len: 912 time: 0.3406541 May 24 11:16:51.324529 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-88ed182b-1711-410e-b672-ddf6d7d5b11c req-06841090-7ed0-4aeb-aa1a-59e03bfeab0e service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=eab44150-44f7-4194-9b76-436d421794e8 HTTP/1.1" status: 200 len: 217 time: 0.1241713 May 24 11:16:51.400192 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:51.550578 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88ed182b-1711-410e-b672-ddf6d7d5b11c req-019a58d3-a3e2-4fc6-af48-ef12f5d8275c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:51.550838 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce62a682-1148-48fb-87cf-a74b9fb3a0d1 req-b0a3c1d8-ba7c-49d7-92cc-5ee80d7ae3e2 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:51.553157 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce62a682-1148-48fb-87cf-a74b9fb3a0d1 req-b0a3c1d8-ba7c-49d7-92cc-5ee80d7ae3e2 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 944 time: 0.1170156 May 24 11:16:51.553873 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-88ed182b-1711-410e-b672-ddf6d7d5b11c req-019a58d3-a3e2-4fc6-af48-ef12f5d8275c service neutron] 10.222.0.93 "GET /v2.0/subnets?id=2fc4be2e-875c-4d10-b857-a48f90c53cf2 HTTP/1.1" status: 200 len: 850 time: 0.2206767 May 24 11:16:51.610260 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce62a682-1148-48fb-87cf-a74b9fb3a0d1 req-d0fb8789-0119-45fb-94ea-3ddf54d9168c tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0507946 May 24 11:16:51.673892 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88ed182b-1711-410e-b672-ddf6d7d5b11c req-5ec5137d-6d0a-4aea-ae1d-99870ebe986d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:51.675971 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-88ed182b-1711-410e-b672-ddf6d7d5b11c req-5ec5137d-6d0a-4aea-ae1d-99870ebe986d service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=1d92641a-d8f9-4ed6-9abd-f464bb26811d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1146197 May 24 11:16:51.915655 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-88ed182b-1711-410e-b672-ddf6d7d5b11c req-b3b7dba5-50a7-4fcf-9241-28051415d1cc service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2317111 May 24 11:16:52.047624 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52930) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:52.103210 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-88ed182b-1711-410e-b672-ddf6d7d5b11c req-33c2143a-3c52-48d2-a7fd-bb49522281cb service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1801600 May 24 11:16:52.239522 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b51c199d-7438-4188-b99c-c2df2184b977 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:52.304903 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-b51c199d-7438-4188-b99c-c2df2184b977 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:52.309842 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-b51c199d-7438-4188-b99c-c2df2184b977 admin admin] Usage tracker for resource:security_group_rule and tenant:c60dc0d7a7a74d309fcdb9a9f49fa177 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:52.317825 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-b51c199d-7438-4188-b99c-c2df2184b977 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:52.328044 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-b51c199d-7438-4188-b99c-c2df2184b977 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:52.380903 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0f4b67b5-f171-49f1-ac32-d760a1dfbb9d req-f3c41686-1e92-4045-9a7f-b17e91af45e4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:52.382695 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0f4b67b5-f171-49f1-ac32-d760a1dfbb9d req-f3c41686-1e92-4045-9a7f-b17e91af45e4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 945 time: 0.1293805 May 24 11:16:52.471497 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0f4b67b5-f171-49f1-ac32-d760a1dfbb9d req-d601c011-1593-41fd-8d2e-02761f12e45a tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0768738 May 24 11:16:52.525456 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b51c199d-7438-4188-b99c-c2df2184b977 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:52.528459 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b51c199d-7438-4188-b99c-c2df2184b977 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:52.571028 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:52.580486 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-b51c199d-7438-4188-b99c-c2df2184b977 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=c60dc0d7a7a74d309fcdb9a9f49fa177&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5317645 May 24 11:16:52.583061 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52952) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:52.606754 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b51c199d-7438-4188-b99c-c2df2184b977 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=578d5ecf-4845-4d5e-ac0e-572a9bf68173,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:52.633331 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b51c199d-7438-4188-b99c-c2df2184b977 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:52.788879 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 52966) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:52.913105 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b53a6076-6b3d-41bf-8e50-b4285a8e1a8b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:52.959660 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b53a6076-6b3d-41bf-8e50-b4285a8e1a8b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:52.960271 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e68bf98e-b210-44e0-b573-2cc70e205ecb admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:52.981355 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e68bf98e-b210-44e0-b573-2cc70e205ecb admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:52.986646 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e68bf98e-b210-44e0-b573-2cc70e205ecb admin admin] Usage tracker for resource:security_group_rule and tenant:ee29a1acf243431b8405240374635a64 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:52.990761 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e68bf98e-b210-44e0-b573-2cc70e205ecb admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:52.998140 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e68bf98e-b210-44e0-b573-2cc70e205ecb admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:53.015458 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b29f698d-acb4-42ca-aeb3-2c0268af7d17 req-2ee3b949-c289-429a-9b1e-e18eebf1a8db tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:53.017032 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b29f698d-acb4-42ca-aeb3-2c0268af7d17 req-2ee3b949-c289-429a-9b1e-e18eebf1a8db tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 944 time: 0.2449775 May 24 11:16:53.048257 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b53a6076-6b3d-41bf-8e50-b4285a8e1a8b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.056296 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:53.078423 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b53a6076-6b3d-41bf-8e50-b4285a8e1a8b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=578d5ecf-4845-4d5e-ac0e-572a9bf68173,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:53.085846 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b53a6076-6b3d-41bf-8e50-b4285a8e1a8b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:53.102116 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b29f698d-acb4-42ca-aeb3-2c0268af7d17 req-12676d03-f680-49e1-bc15-7844b93e7090 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0742068 May 24 11:16:53.103076 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b53a6076-6b3d-41bf-8e50-b4285a8e1a8b admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.104208 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-b53a6076-6b3d-41bf-8e50-b4285a8e1a8b admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/578d5ecf-4845-4d5e-ac0e-572a9bf68173 HTTP/1.1" status: 204 len: 168 time: 0.5197332 May 24 11:16:53.124851 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e68bf98e-b210-44e0-b573-2cc70e205ecb admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.129200 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e68bf98e-b210-44e0-b573-2cc70e205ecb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.139013 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ca6b0133-e01b-4106-8435-4a8cb13e5d1b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:53.184226 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e68bf98e-b210-44e0-b573-2cc70e205ecb admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=ee29a1acf243431b8405240374635a64&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3916099 May 24 11:16:53.185282 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e68bf98e-b210-44e0-b573-2cc70e205ecb admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=487132ad-2428-4ba2-be5c-a37d9b6e6cd6,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:53.187584 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52974) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:53.190588 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e68bf98e-b210-44e0-b573-2cc70e205ecb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:53.256274 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6467465b-0b2c-4a37-9e4c-496171fb1c5d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.285210 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6467465b-0b2c-4a37-9e4c-496171fb1c5d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.356817 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6467465b-0b2c-4a37-9e4c-496171fb1c5d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.362340 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f7f60d14-f1ba-4258-80cd-730c4922371f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:53.379099 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6467465b-0b2c-4a37-9e4c-496171fb1c5d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=487132ad-2428-4ba2-be5c-a37d9b6e6cd6,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:53.382587 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6467465b-0b2c-4a37-9e4c-496171fb1c5d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:53.383688 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6467465b-0b2c-4a37-9e4c-496171fb1c5d admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.385197 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-6467465b-0b2c-4a37-9e4c-496171fb1c5d admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/487132ad-2428-4ba2-be5c-a37d9b6e6cd6 HTTP/1.1" status: 204 len: 168 time: 0.1959543 May 24 11:16:53.424775 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-7932635d-2f52-4a02-baa1-ff0ae8f18838 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.454918 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device eab44150-44f7-4194-9b76-436d421794e8 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:16:53.457657 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 692ab816-0fdb-448b-9b27-c44c0a457f4d up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:53.508802 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52986) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:53.532871 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0f90a29c-0790-4713-be80-dec540dd0df4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.566682 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-0f90a29c-0790-4713-be80-dec540dd0df4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:53.574203 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-0f90a29c-0790-4713-be80-dec540dd0df4 admin admin] Usage tracker for resource:security_group_rule and tenant:71c6afd69b174e6684ae669ad8b79c68 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:53.578137 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-0f90a29c-0790-4713-be80-dec540dd0df4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:53.590469 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-0f90a29c-0790-4713-be80-dec540dd0df4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:53.727786 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 692ab816-0fdb-448b-9b27-c44c0a457f4d completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:53.737037 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 692ab816-0fdb-448b-9b27-c44c0a457f4d triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:53.737846 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.750865 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0f90a29c-0790-4713-be80-dec540dd0df4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.756593 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0f90a29c-0790-4713-be80-dec540dd0df4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.770323 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e6b58333-8b71-4626-b4de-3c16765c2cc5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:53.836426 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-0f90a29c-0790-4713-be80-dec540dd0df4 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=71c6afd69b174e6684ae669ad8b79c68&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3253553 May 24 11:16:53.837130 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 52998) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:53.840856 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port eab44150-44f7-4194-9b76-436d421794e8, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=eab44150-44f7-4194-9b76-436d421794e8,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:53.844926 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-144b35f6-bbf7-4dca-8693-300d57ca37e7 req-78f80248-4093-4e1b-9466-73ea336860cc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:53.846760 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-144b35f6-bbf7-4dca-8693-300d57ca37e7 req-78f80248-4093-4e1b-9466-73ea336860cc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 945 time: 0.1237037 May 24 11:16:53.865806 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': 'eab44150-44f7-4194-9b76-436d421794e8', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'port_id': 'eab44150-44f7-4194-9b76-436d421794e8', 'mac_address': 'fa:16:3e:b2:cf:de', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 518, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:16:53.908925 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-144b35f6-bbf7-4dca-8693-300d57ca37e7 req-52a57a3a-4a9b-4729-887d-1cae2bb737b9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0541589 May 24 11:16:53.915037 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53004) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:53.920780 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0f90a29c-0790-4713-be80-dec540dd0df4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=fb27b09b-5db9-4db1-95e5-b24490f2bd40,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:53.940810 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0f90a29c-0790-4713-be80-dec540dd0df4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.170s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:53.958791 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-710449fd-ea6b-4377-b7ab-19ed9f1f65c4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.977164 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-710449fd-ea6b-4377-b7ab-19ed9f1f65c4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:53.983207 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-710449fd-ea6b-4377-b7ab-19ed9f1f65c4 admin admin] Usage tracker for resource:security_group_rule and tenant:23e8c3e7bcf746379adac94a3e8c9245 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:53.984788 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-598b9176-1b54-4763-9808-afd6361557aa admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:53.987538 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-710449fd-ea6b-4377-b7ab-19ed9f1f65c4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:53.994888 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-710449fd-ea6b-4377-b7ab-19ed9f1f65c4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:54.028058 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-598b9176-1b54-4763-9808-afd6361557aa admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.065009 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.085381 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.104286 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-598b9176-1b54-4763-9808-afd6361557aa admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.111412 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-4638d1b2-69d8-4662-929f-0c56286d9729 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:54.117350 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id eab44150-44f7-4194-9b76-436d421794e8 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:16:54.136901 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-598b9176-1b54-4763-9808-afd6361557aa admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=fb27b09b-5db9-4db1-95e5-b24490f2bd40,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:54.139301 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-598b9176-1b54-4763-9808-afd6361557aa admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:54.140810 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-598b9176-1b54-4763-9808-afd6361557aa admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.143801 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 692ab816-0fdb-448b-9b27-c44c0a457f4d, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=692ab816-0fdb-448b-9b27-c44c0a457f4d,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:54.145040 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-598b9176-1b54-4763-9808-afd6361557aa admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/fb27b09b-5db9-4db1-95e5-b24490f2bd40 HTTP/1.1" status: 204 len: 168 time: 0.3051782 May 24 11:16:54.145462 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'ef0f174b-4c60-4917-8c38-60dd5c2c0add', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '692ab816-0fdb-448b-9b27-c44c0a457f4d'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:16:54.157343 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.158439 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 02210ade-8c60-41b6-aa82-a104d4fc0447 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:54.159548 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-710449fd-ea6b-4377-b7ab-19ed9f1f65c4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.167589 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-710449fd-ea6b-4377-b7ab-19ed9f1f65c4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.189056 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:54.212199 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-1880bc9c-6fcb-4473-96ef-8c433337f7cd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:54.266723 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'ef0f174b-4c60-4917-8c38-60dd5c2c0add', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '692ab816-0fdb-448b-9b27-c44c0a457f4d', 'code': 200} May 24 11:16:54.276299 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-710449fd-ea6b-4377-b7ab-19ed9f1f65c4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7df56a5e-f581-4e15-91d4-12e1a070071e,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:54.304199 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-710449fd-ea6b-4377-b7ab-19ed9f1f65c4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:54.327054 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-710449fd-ea6b-4377-b7ab-19ed9f1f65c4 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=23e8c3e7bcf746379adac94a3e8c9245&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4056048 May 24 11:16:54.327808 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53026) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:54.341304 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port eab44150-44f7-4194-9b76-436d421794e8, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=eab44150-44f7-4194-9b76-436d421794e8,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:54.399552 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.414061 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=692ab816-0fdb-448b-9b27-c44c0a457f4d,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:54.422655 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.210s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:54.425513 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.019s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:54.516280 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 02210ade-8c60-41b6-aa82-a104d4fc0447 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:54.523918 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-548c3748-a63a-459c-87a0-d83043ef6980 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.530228 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 02210ade-8c60-41b6-aa82-a104d4fc0447 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:54.531133 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.606990 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-548c3748-a63a-459c-87a0-d83043ef6980 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.636607 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fc002d14-4ea7-4b98-9389-2fe11120b6c1 req-b4eeb9f1-dc7a-4fa4-9005-e73917243661 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:54.639161 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-fc002d14-4ea7-4b98-9389-2fe11120b6c1 req-b4eeb9f1-dc7a-4fa4-9005-e73917243661 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 945 time: 0.2017455 May 24 11:16:54.660168 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=eab44150-44f7-4194-9b76-436d421794e8,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:54.664856 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.239s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:54.703240 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-fc002d14-4ea7-4b98-9389-2fe11120b6c1 req-17c66c8f-f5e9-4f7f-b69e-3cbb1d7333e7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0579405 May 24 11:16:54.758846 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['eab44150-44f7-4194-9b76-436d421794e8'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:16:54.759218 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6a35f28-9582-48a0-a1a3-f29451da5658 req-1b531ce2-5d70-49d2-81f7-a883f8156941 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:54.760697 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a6a35f28-9582-48a0-a1a3-f29451da5658 req-1b531ce2-5d70-49d2-81f7-a883f8156941 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 1099 time: 0.1360633 May 24 11:16:54.842748 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-548c3748-a63a-459c-87a0-d83043ef6980 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.850470 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.865965 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:54.897655 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-548c3748-a63a-459c-87a0-d83043ef6980 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7df56a5e-f581-4e15-91d4-12e1a070071e,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:54.900655 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-548c3748-a63a-459c-87a0-d83043ef6980 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.902204 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-548c3748-a63a-459c-87a0-d83043ef6980 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/7df56a5e-f581-4e15-91d4-12e1a070071e HTTP/1.1" status: 204 len: 168 time: 0.5725112 May 24 11:16:54.904154 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-548c3748-a63a-459c-87a0-d83043ef6980 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:54.918734 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53040) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:54.925127 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 02210ade-8c60-41b6-aa82-a104d4fc0447, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=02210ade-8c60-41b6-aa82-a104d4fc0447,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:54.941736 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:54.946551 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a6a35f28-9582-48a0-a1a3-f29451da5658 req-4abc6016-4b26-46f7-8e32-55592e182a71 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1748788 May 24 11:16:54.949034 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:55.080800 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=02210ade-8c60-41b6-aa82-a104d4fc0447,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:55.091715 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.143s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:55.190515 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53056) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:55.218167 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-79c981ee-3995-4623-82c4-8ce7b8a1ba29 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.243040 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d4712916-9fd6-43d6-8e56-ddc24a70a55c req-8369bbbf-1ba3-4d9e-8409-722c585e7565 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:55.250095 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-79c981ee-3995-4623-82c4-8ce7b8a1ba29 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:55.257887 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d4712916-9fd6-43d6-8e56-ddc24a70a55c req-8369bbbf-1ba3-4d9e-8409-722c585e7565 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1534786 May 24 11:16:55.262025 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-79c981ee-3995-4623-82c4-8ce7b8a1ba29 admin admin] Usage tracker for resource:security_group_rule and tenant:1d9d5f4c966f45a3a80bafb114fb7df9 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:55.268306 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-79c981ee-3995-4623-82c4-8ce7b8a1ba29 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:55.282067 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-79c981ee-3995-4623-82c4-8ce7b8a1ba29 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:55.325733 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d4712916-9fd6-43d6-8e56-ddc24a70a55c req-52152bbf-527d-48fa-8db2-205916035bc1 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0667813 May 24 11:16:55.379626 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2fe50351-31e9-41d7-935c-5d14bbd52f32 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.411127 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-2fe50351-31e9-41d7-935c-5d14bbd52f32 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:55.423315 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-2fe50351-31e9-41d7-935c-5d14bbd52f32 admin admin] Usage tracker for resource:security_group_rule and tenant:f7bb0a239015468eaf7cb9ac75c5a709 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:55.430489 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-2fe50351-31e9-41d7-935c-5d14bbd52f32 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:55.432877 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-79c981ee-3995-4623-82c4-8ce7b8a1ba29 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.437209 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-2fe50351-31e9-41d7-935c-5d14bbd52f32 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:55.440621 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-79c981ee-3995-4623-82c4-8ce7b8a1ba29 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.450277 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:55.506268 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-79c981ee-3995-4623-82c4-8ce7b8a1ba29 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=1d9d5f4c966f45a3a80bafb114fb7df9&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3110893 May 24 11:16:55.506853 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53068) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:55.518807 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-79c981ee-3995-4623-82c4-8ce7b8a1ba29 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=140c6dd9-8874-4c9a-84a3-200f8186f722,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:55.524211 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-79c981ee-3995-4623-82c4-8ce7b8a1ba29 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:55.604429 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2fe50351-31e9-41d7-935c-5d14bbd52f32 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.609742 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2fe50351-31e9-41d7-935c-5d14bbd52f32 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.615062 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8034e5fb-dd04-41da-8d90-e164c16f56ff admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.620883 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-598b9176-1b54-4763-9808-afd6361557aa admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:55.650810 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8034e5fb-dd04-41da-8d90-e164c16f56ff admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.683038 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2fe50351-31e9-41d7-935c-5d14bbd52f32 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=981f28e5-51a5-4bfd-a714-6bd0397f6cb1,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:55.690425 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-2fe50351-31e9-41d7-935c-5d14bbd52f32 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:55.694564 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-2fe50351-31e9-41d7-935c-5d14bbd52f32 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=f7bb0a239015468eaf7cb9ac75c5a709&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7736282 May 24 11:16:55.697604 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53076) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:55.723102 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8034e5fb-dd04-41da-8d90-e164c16f56ff admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.734558 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:55.753392 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8034e5fb-dd04-41da-8d90-e164c16f56ff admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=140c6dd9-8874-4c9a-84a3-200f8186f722,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:55.757383 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8034e5fb-dd04-41da-8d90-e164c16f56ff admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.759305 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-8034e5fb-dd04-41da-8d90-e164c16f56ff admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/140c6dd9-8874-4c9a-84a3-200f8186f722 HTTP/1.1" status: 204 len: 168 time: 0.2486176 May 24 11:16:55.760703 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8034e5fb-dd04-41da-8d90-e164c16f56ff admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:55.763247 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.828870 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9a872824-8cb2-4bf1-8042-f88ec51bbeda admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.862077 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-7c10c901-e0a6-49a1-84d6-288e0aa374b5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:55.873540 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9a872824-8cb2-4bf1-8042-f88ec51bbeda admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:56.019127 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9fcaa20b-eeb7-439a-8076-6a5985e55b90 req-b26045e5-ee05-42be-bbf8-af6d7910d8ba tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:56.027928 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9fcaa20b-eeb7-439a-8076-6a5985e55b90 req-b26045e5-ee05-42be-bbf8-af6d7910d8ba tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 945 time: 0.1296182 May 24 11:16:56.082205 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9a872824-8cb2-4bf1-8042-f88ec51bbeda admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:56.088077 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b53a6076-6b3d-41bf-8e50-b4285a8e1a8b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:56.107376 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9fcaa20b-eeb7-439a-8076-6a5985e55b90 req-7ccd40bd-dc4a-41d9-ab38-fabac7653ce5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0746598 May 24 11:16:56.116553 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9a872824-8cb2-4bf1-8042-f88ec51bbeda admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=981f28e5-51a5-4bfd-a714-6bd0397f6cb1,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:56.121523 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-9a872824-8cb2-4bf1-8042-f88ec51bbeda admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:56.127194 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9a872824-8cb2-4bf1-8042-f88ec51bbeda admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:56.130773 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-9a872824-8cb2-4bf1-8042-f88ec51bbeda admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/981f28e5-51a5-4bfd-a714-6bd0397f6cb1 HTTP/1.1" status: 204 len: 168 time: 0.4296079 May 24 11:16:56.269043 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'd9e36d1d-343c-4b97-b226-9b58722aebb2', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '02210ade-8c60-41b6-aa82-a104d4fc0447'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:16:56.289245 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:16:56.360136 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'd9e36d1d-343c-4b97-b226-9b58722aebb2', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '02210ade-8c60-41b6-aa82-a104d4fc0447', 'code': 200} May 24 11:16:56.694396 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce4d98c0-2ea1-4684-ae89-07374a0aa9de req-08a7c470-8655-485b-abd4-776cb586340b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:56.696434 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce4d98c0-2ea1-4684-ae89-07374a0aa9de req-08a7c470-8655-485b-abd4-776cb586340b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1669157 May 24 11:16:56.781747 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce4d98c0-2ea1-4684-ae89-07374a0aa9de req-e59afc85-838b-4576-9453-512d2583b636 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0767462 May 24 11:16:56.800156 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7ca1c250-ff81-40c3-8acb-277bb91e47bc req-b42e645f-5fcc-4a89-9130-7d4b19eb3395 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:56.801639 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7ca1c250-ff81-40c3-8acb-277bb91e47bc req-b42e645f-5fcc-4a89-9130-7d4b19eb3395 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 1099 time: 0.1611838 May 24 11:16:56.907202 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7ca1c250-ff81-40c3-8acb-277bb91e47bc req-9a9d4193-2f49-466c-8d7c-79eb3644792c service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0857861 May 24 11:16:56.955284 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53132) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:56.961392 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:56.978099 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:56.996022 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:57.000350 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Usage tracker for resource:security_group_rule and tenant:e50160a1d9274614939a1ce11f34fc6b is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:57.003503 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:57.010785 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:57.134699 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:57.142560 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:57.153895 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f023d0c0-e305-4995-9579-4f8bf676fb3f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:57.209603 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=75328fd6-bd98-46fa-bbac-a098c03ad2cd,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:57.218006 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53144) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:57.218169 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=e50160a1d9274614939a1ce11f34fc6b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2617486 May 24 11:16:57.219943 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.220297 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.220745 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.220969 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.221329 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.221668 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.222194 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.223109 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.223503 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.224332 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.224720 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.225044 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.229334 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:57.335603 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8ea8b1a-6640-449f-9499-d027d95f7763 req-7c59ef03-dffb-473c-9339-8fd805d1dbe3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:57.338549 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8ea8b1a-6640-449f-9499-d027d95f7763 req-7c59ef03-dffb-473c-9339-8fd805d1dbe3 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 1099 time: 0.0871792 May 24 11:16:57.417401 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:57.473071 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:57.475659 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device eab44150-44f7-4194-9b76-436d421794e8 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:16:57.650036 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0abe8911-a93a-4adc-b3f1-5f9e929916d2 req-579adba1-5a30-446e-803f-fee60e178c8c tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:57.653134 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0abe8911-a93a-4adc-b3f1-5f9e929916d2 req-579adba1-5a30-446e-803f-fee60e178c8c tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 945 time: 0.3038085 May 24 11:16:57.667447 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port eab44150-44f7-4194-9b76-436d421794e8 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:16:57.678580 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port eab44150-44f7-4194-9b76-436d421794e8 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:16:57.679108 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:57.747256 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8ea8b1a-6640-449f-9499-d027d95f7763 req-2fbff01e-977e-4021-8f87-515dd21ca851 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:57.755317 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8ea8b1a-6640-449f-9499-d027d95f7763 req-2fbff01e-977e-4021-8f87-515dd21ca851 service neutron] 10.222.0.93 "GET /v2.0/networks?id=4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 200 len: 918 time: 0.3975267 May 24 11:16:57.767040 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53162) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:57.789492 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:57.809017 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0abe8911-a93a-4adc-b3f1-5f9e929916d2 req-711d2fd7-243a-43e1-9d13-7a43b72e12c3 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1367061 May 24 11:16:57.823037 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-2fe50351-31e9-41d7-935c-5d14bbd52f32 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:57.848144 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=75328fd6-bd98-46fa-bbac-a098c03ad2cd,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:57.866379 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:57.877658 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8ea8b1a-6640-449f-9499-d027d95f7763 req-d7bc43e0-9e20-4b83-bf27-bd5121d6783b service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=692ab816-0fdb-448b-9b27-c44c0a457f4d HTTP/1.1" status: 200 len: 217 time: 0.1114094 May 24 11:16:57.879495 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/75328fd6-bd98-46fa-bbac-a098c03ad2cd HTTP/1.1" status: 204 len: 168 time: 0.6599398 May 24 11:16:57.881918 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.882558 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.883116 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.883672 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.884213 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.884752 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.885332 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.885885 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.890973 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.891641 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.892213 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.892791 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:16:57.900772 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:57.960540 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:58.033025 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-33ecdf14-351a-4a81-b666-cb8b53534cbc req-6a53e91b-5f3e-4648-a514-4ba41d5c9134 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:58.034572 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-33ecdf14-351a-4a81-b666-cb8b53534cbc req-6a53e91b-5f3e-4648-a514-4ba41d5c9134 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.6458304 May 24 11:16:58.061580 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port eab44150-44f7-4194-9b76-436d421794e8, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=eab44150-44f7-4194-9b76-436d421794e8,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:16:58.069462 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:58.069617 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-764031376-network', 'tenant_id': '0dc56b1d592a4beea17c8068d8779718'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:58.075368 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-1969920a-de31-4b65-a8f3-d82eb3bfd2c9 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:58.082020 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:58.094031 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Usage tracker for resource:network and tenant:0dc56b1d592a4beea17c8068d8779718 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:58.116754 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:58.142134 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:58.181272 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Made reservation on behalf of 0dc56b1d592a4beea17c8068d8779718 for: {'network': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:58.183698 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:58.226518 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:58.232510 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=eab44150-44f7-4194-9b76-436d421794e8,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:58.238881 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.164s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:58.261205 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8ea8b1a-6640-449f-9499-d027d95f7763 req-8fed6192-553c-4174-9393-f8e4d366277c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:58.262622 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8ea8b1a-6640-449f-9499-d027d95f7763 req-8fed6192-553c-4174-9393-f8e4d366277c service neutron] 10.222.0.93 "GET /v2.0/subnets?id=52267672-fd8c-45bd-89ea-d9d02a7b249a HTTP/1.1" status: 200 len: 857 time: 0.3751543 May 24 11:16:58.270579 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-33ecdf14-351a-4a81-b666-cb8b53534cbc req-ebec4e6f-330c-4430-9046-dcc5f15fec78 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2157769 May 24 11:16:58.278101 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:58.300317 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Usage tracker for resource:security_group_rule and tenant:0dc56b1d592a4beea17c8068d8779718 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:58.307164 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:58.322882 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:58.361142 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8ea8b1a-6640-449f-9499-d027d95f7763 req-a0aa505b-865c-4d09-8c33-ff6129f18ae0 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=4fec6b68-64f1-4553-8ca5-f86c63ac9714&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0683799 May 24 11:16:58.361542 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'eab44150-44f7-4194-9b76-436d421794e8'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:16:58.463407 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'eab44150-44f7-4194-9b76-436d421794e8', 'code': 200} May 24 11:16:58.504152 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:58.511158 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:58.520179 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:58.564530 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=49092711-990a-4d12-82d5-1ac9ef151ab3,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:58.583590 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:58.595214 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] vlan segment allocate from pool success with {'vlan_id': 526, 'physical_network': 'physnet1'} {{(pid=91820) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:16:58.607069 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:58.607762 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.segments_db [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Added segment beb69048-046c-49d6-a095-eb202a63ca1d of type vlan for network 98ea8460-214e-4f24-a638-0313bfc40de5 May 24 11:16:58.654262 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:58.670624 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8ea8b1a-6640-449f-9499-d027d95f7763 req-8707dbb1-950a-44c1-a88b-f389f0e32359 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2874773 May 24 11:16:58.705331 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:58.707111 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:58.707907 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:58.717346 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b6e2bf1f-b9b7-4eff-81af-65eb3398fa47 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:58.880089 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8ea8b1a-6640-449f-9499-d027d95f7763 req-c5564e5a-e7ea-4903-a465-d4c46cbd0855 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1931367 May 24 11:16:58.892359 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:58.996968 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 1.2250471 May 24 11:16:59.001047 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53220) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:16:59.008471 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Pushing event updated for resources: {'Network': ['ID=98ea8460-214e-4f24-a638-0313bfc40de5,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:59.019565 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network 98ea8460-214e-4f24-a638-0313bfc40de5 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:16:59.020405 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Request body: {'subnet': {'network_id': '98ea8460-214e-4f24-a638-0313bfc40de5', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-764031376-subnet', 'tenant_id': '0dc56b1d592a4beea17c8068d8779718', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:16:59.026444 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.308s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:59.029991 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:16:59.039191 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Usage tracker for resource:subnet and tenant:0dc56b1d592a4beea17c8068d8779718 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:16:59.043171 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:16:59.051218 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:16:59.059644 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ebc5108f-e2ad-4530-8d33-f789bda1de8d req-4c730071-3f30-4152-bd81-0b4f8ac91ea7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:59.060979 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-ebc5108f-e2ad-4530-8d33-f789bda1de8d req-4c730071-3f30-4152-bd81-0b4f8ac91ea7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Synchronizing usage tracker for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:16:59.062798 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d7ac1d43-49fa-4a30-a590-0cf4280701a1 req-ad178d96-f334-4930-93fe-8d65c7f7375a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:59.064524 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d7ac1d43-49fa-4a30-a590-0cf4280701a1 req-ad178d96-f334-4930-93fe-8d65c7f7375a service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 1099 time: 0.2958720 May 24 11:16:59.070193 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Made reservation on behalf of 0dc56b1d592a4beea17c8068d8779718 for: {'subnet': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:16:59.073276 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-535456753'] for subnet, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:59.081078 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-ebc5108f-e2ad-4530-8d33-f789bda1de8d req-4c730071-3f30-4152-bd81-0b4f8ac91ea7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Unset dirty status for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:16:59.082509 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ebc5108f-e2ad-4530-8d33-f789bda1de8d req-4c730071-3f30-4152-bd81-0b4f8ac91ea7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1224709 May 24 11:16:59.158081 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ebc5108f-e2ad-4530-8d33-f789bda1de8d req-172231cc-6282-4e2e-8740-ff7375d13d10 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0690365 May 24 11:16:59.586466 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-767d78af-9387-4e2e-9896-10023d83b345 req-a04f56a5-10a2-467e-9d8f-cd42e698c4dc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:59.593267 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-767d78af-9387-4e2e-9896-10023d83b345 req-a04f56a5-10a2-467e-9d8f-cd42e698c4dc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1799333 May 24 11:16:59.648714 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-767d78af-9387-4e2e-9896-10023d83b345 req-4b2f3f51-0f6d-4d31-a0b2-fdcb9a01afec tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0545144 May 24 11:16:59.748902 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-534788555', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-535456743'] for subnet, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:59.753082 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:16:59.826939 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Pushing event updated for resources: {'Subnet': ['ID=d160e17b-87b6-4e4e-a58c-1a5ce3d62fd4,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:16:59.830870 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:16:59.837543 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Checking segment: {'id': 'beb69048-046c-49d6-a095-eb202a63ca1d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 526, 'network_id': '98ea8460-214e-4f24-a638-0313bfc40de5'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:16:59.837891 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Network hostable DHCP agents. Network: 98ea8460-214e-4f24-a638-0313bfc40de5, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91820) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:16:59.856636 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Network 98ea8460-214e-4f24-a638-0313bfc40de5 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91820) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:16:59.869179 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, 98ea8460-214e-4f24-a638-0313bfc40de5) {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:16:59.880529 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:16:59.881445 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:16:59.886838 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-b9b419d1-5735-4f72-8e1a-7c287c4db431 None None] Network 98ea8460-214e-4f24-a638-0313bfc40de5 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:16:59.909359 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 0.9030986 May 24 11:17:00.127855 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-1993d394-6f2c-4611-af3c-5dcc5fb1b72d None None] Network 98ea8460-214e-4f24-a638-0313bfc40de5 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:17:00.406703 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53274) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:00.453425 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ddff100c-3950-4f8e-aa0d-6e683c981293 req-40ce7dea-3b86-4cb2-9fcb-feb5909e25e8 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:00.460638 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ddff100c-3950-4f8e-aa0d-6e683c981293 req-40ce7dea-3b86-4cb2-9fcb-feb5909e25e8 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1143279 May 24 11:17:00.520896 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ddff100c-3950-4f8e-aa0d-6e683c981293 req-567abe19-7661-40ea-b66b-2cb63cd11855 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0590518 May 24 11:17:00.653302 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df59f3fe-46f4-426d-9e08-8a4b377ff232 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:00.669570 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-df59f3fe-46f4-426d-9e08-8a4b377ff232 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:00.680276 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-df59f3fe-46f4-426d-9e08-8a4b377ff232 admin admin] Usage tracker for resource:security_group_rule and tenant:1c61a9d0ba924be996f43f4813d658d2 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:00.685506 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-df59f3fe-46f4-426d-9e08-8a4b377ff232 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:00.693299 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-df59f3fe-46f4-426d-9e08-8a4b377ff232 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:00.862222 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df59f3fe-46f4-426d-9e08-8a4b377ff232 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:00.868769 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df59f3fe-46f4-426d-9e08-8a4b377ff232 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:00.877857 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:00.932515 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-df59f3fe-46f4-426d-9e08-8a4b377ff232 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=1c61a9d0ba924be996f43f4813d658d2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5221114 May 24 11:17:00.932515 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-df59f3fe-46f4-426d-9e08-8a4b377ff232 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=86b9fc4c-db1c-48e9-b24e-011fc4ccc783,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:00.932800 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53304) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:00.932800 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53306) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:00.935641 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-df59f3fe-46f4-426d-9e08-8a4b377ff232 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:00.941828 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-2094666883-network', 'tenant_id': '3e4e2c9d116d40dc825ed63a0bf2f2d1'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:17:00.947348 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0e6414f-61fe-4a54-b74d-6b3812f66074 req-d0c81d52-5dd6-4d7a-9a45-264960440c55 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:00.949178 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d0e6414f-61fe-4a54-b74d-6b3812f66074 req-d0c81d52-5dd6-4d7a-9a45-264960440c55 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1746891 May 24 11:17:00.965304 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:00.972568 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Usage tracker for resource:network and tenant:3e4e2c9d116d40dc825ed63a0bf2f2d1 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:00.976950 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:00.995644 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:01.024915 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Made reservation on behalf of 3e4e2c9d116d40dc825ed63a0bf2f2d1 for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:17:01.026990 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d0e6414f-61fe-4a54-b74d-6b3812f66074 req-de55dbce-a4fb-406d-b668-bfa392d04673 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0718980 May 24 11:17:01.030577 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.036698 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d7ac1d43-49fa-4a30-a590-0cf4280701a1 req-5ec36bc0-c4f5-4e44-9ed1-dcc1d5c94a08 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0624933 May 24 11:17:01.057519 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.071110 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c127b25b-8696-4f27-a058-b6a8c2779865 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.087354 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:01.099421 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Usage tracker for resource:security_group_rule and tenant:3e4e2c9d116d40dc825ed63a0bf2f2d1 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:01.104559 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:01.113197 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c127b25b-8696-4f27-a058-b6a8c2779865 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.119585 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:01.207153 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c127b25b-8696-4f27-a058-b6a8c2779865 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.216221 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8034e5fb-dd04-41da-8d90-e164c16f56ff admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:01.232387 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c127b25b-8696-4f27-a058-b6a8c2779865 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=86b9fc4c-db1c-48e9-b24e-011fc4ccc783,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:01.235306 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c127b25b-8696-4f27-a058-b6a8c2779865 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:01.275379 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.276678 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c127b25b-8696-4f27-a058-b6a8c2779865 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.278174 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-c127b25b-8696-4f27-a058-b6a8c2779865 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/86b9fc4c-db1c-48e9-b24e-011fc4ccc783 HTTP/1.1" status: 204 len: 168 time: 0.3425395 May 24 11:17:01.279461 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.283022 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e68bf98e-b210-44e0-b573-2cc70e205ecb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:01.316087 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] vlan segment allocate from pool success with {'vlan_id': 531, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:17:01.321543 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=18863090-0523-4364-bad8-a74b1bc86112,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:01.325118 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.042s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:01.330685 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.331218 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Added segment 140422ff-fbc9-4c7d-874b-339405af2dc7 of type vlan for network c24a20e8-37d4-4a0c-b252-5fc58f34aac1 May 24 11:17:01.363412 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.399177 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.401139 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:01.401777 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.407000 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:01.516039 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 0.5821538 May 24 11:17:01.519079 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53336) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:01.521247 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Pushing event updated for resources: {'Network': ['ID=c24a20e8-37d4-4a0c-b252-5fc58f34aac1,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:01.524453 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.118s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:01.536242 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network c24a20e8-37d4-4a0c-b252-5fc58f34aac1 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:17:01.546934 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Request body: {'subnet': {'network_id': 'c24a20e8-37d4-4a0c-b252-5fc58f34aac1', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-2094666883-subnet', 'tenant_id': '3e4e2c9d116d40dc825ed63a0bf2f2d1', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:17:01.564521 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:01.578175 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Usage tracker for resource:subnet and tenant:3e4e2c9d116d40dc825ed63a0bf2f2d1 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:01.585255 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:01.597257 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:01.629339 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Made reservation on behalf of 3e4e2c9d116d40dc825ed63a0bf2f2d1 for: {'subnet': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:17:01.634789 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-535456753'] for subnet, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.798174 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53348) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:01.821137 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5b6ef236-f47e-480e-a2e1-1d98ceb7f12b admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:01.840090 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-5b6ef236-f47e-480e-a2e1-1d98ceb7f12b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:01.845858 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-5b6ef236-f47e-480e-a2e1-1d98ceb7f12b admin admin] Usage tracker for resource:security_group_rule and tenant:5e6d4a9236de40d0b0b2fa3c677c4648 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:01.852219 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-5b6ef236-f47e-480e-a2e1-1d98ceb7f12b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:01.866044 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dd4e2042-974b-4c57-9843-d9735e17f7b2 req-b3f0147d-079e-4cc7-8b29-20a3d83b84e1 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:01.869855 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dd4e2042-974b-4c57-9843-d9735e17f7b2 req-b3f0147d-079e-4cc7-8b29-20a3d83b84e1 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1606832 May 24 11:17:01.871725 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-5b6ef236-f47e-480e-a2e1-1d98ceb7f12b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:01.941038 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dd4e2042-974b-4c57-9843-d9735e17f7b2 req-b7ba0c80-115f-4a0c-b99d-8a057573883a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0664821 May 24 11:17:01.981008 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53354) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:02.021987 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5b6ef236-f47e-480e-a2e1-1d98ceb7f12b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.026922 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5b6ef236-f47e-480e-a2e1-1d98ceb7f12b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.036713 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b2e8ebe9-db23-4e9a-ad31-8f371b2ab49a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:02.082514 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-5b6ef236-f47e-480e-a2e1-1d98ceb7f12b admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=5e6d4a9236de40d0b0b2fa3c677c4648&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2828021 May 24 11:17:02.084250 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53362) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:02.087509 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b6ef236-f47e-480e-a2e1-1d98ceb7f12b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=af862f17-6d8b-4821-977e-191b46f1b72c,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:02.119283 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5b6ef236-f47e-480e-a2e1-1d98ceb7f12b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:02.130733 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-534788555', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-535456743'] for subnet, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.152219 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:02.153431 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d967634a-43b7-44a0-b634-e3bfefac66b3 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.177622 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-47e4eb83-47c3-440f-a48b-2b3036f76bbf admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.180663 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d967634a-43b7-44a0-b634-e3bfefac66b3 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.246095 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d967634a-43b7-44a0-b634-e3bfefac66b3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.250931 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-250de9b1-b2f1-4647-9ed1-82be32d9501a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:02.264151 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-47e4eb83-47c3-440f-a48b-2b3036f76bbf admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:02.267093 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d967634a-43b7-44a0-b634-e3bfefac66b3 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=af862f17-6d8b-4821-977e-191b46f1b72c,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:02.270008 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d967634a-43b7-44a0-b634-e3bfefac66b3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:02.272687 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d967634a-43b7-44a0-b634-e3bfefac66b3 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.276515 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-d967634a-43b7-44a0-b634-e3bfefac66b3 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/af862f17-6d8b-4821-977e-191b46f1b72c HTTP/1.1" status: 204 len: 168 time: 0.1911685 May 24 11:17:02.284425 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-47e4eb83-47c3-440f-a48b-2b3036f76bbf admin admin] Usage tracker for resource:security_group_rule and tenant:49c8f6213b1f425c88533cf94c9246c5 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:02.298583 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-47e4eb83-47c3-440f-a48b-2b3036f76bbf admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:02.315598 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-47e4eb83-47c3-440f-a48b-2b3036f76bbf admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:02.335230 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Pushing event updated for resources: {'Subnet': ['ID=51c7b9d7-d7c7-4132-b656-203c2f9e230a,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:02.341879 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.190s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:02.352659 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Checking segment: {'id': '140422ff-fbc9-4c7d-874b-339405af2dc7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 531, 'network_id': 'c24a20e8-37d4-4a0c-b252-5fc58f34aac1'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:17:02.353343 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Network hostable DHCP agents. Network: c24a20e8-37d4-4a0c-b252-5fc58f34aac1, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91820) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:17:02.415321 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Network c24a20e8-37d4-4a0c-b252-5fc58f34aac1 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91820) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:17:02.433919 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-775396ab-cec9-4d94-99ab-424920e5fd9e req-f1e43459-9cd4-4461-ae41-6ac0397ce2db tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:02.435572 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-775396ab-cec9-4d94-99ab-424920e5fd9e req-f1e43459-9cd4-4461-ae41-6ac0397ce2db tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.2489052 May 24 11:17:02.465840 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, c24a20e8-37d4-4a0c-b252-5fc58f34aac1) {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:17:02.479544 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c1d8d63b-a319-4033-8137-04b5ebbed2af None None] Network c24a20e8-37d4-4a0c-b252-5fc58f34aac1 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:17:02.498568 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:02.499053 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.525858 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-47e4eb83-47c3-440f-a48b-2b3036f76bbf admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.544490 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-775396ab-cec9-4d94-99ab-424920e5fd9e req-3cd164dc-5c70-47c1-aaf2-600c4dc040fc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0926628 May 24 11:17:02.550418 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-47e4eb83-47c3-440f-a48b-2b3036f76bbf admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.572386 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-79c981ee-3995-4623-82c4-8ce7b8a1ba29 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:02.613146 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 846 time: 1.0885351 May 24 11:17:02.628908 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-47e4eb83-47c3-440f-a48b-2b3036f76bbf admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=49c8f6213b1f425c88533cf94c9246c5&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6428344 May 24 11:17:02.629078 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53378) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:02.665778 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-47e4eb83-47c3-440f-a48b-2b3036f76bbf admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1e88b4a0-fe96-459c-a8f3-3e43be9a4979,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:02.697160 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-47e4eb83-47c3-440f-a48b-2b3036f76bbf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.125s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:02.787013 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-a2c6c047-25b5-4eac-83bc-65ceed53ee2e None None] Network c24a20e8-37d4-4a0c-b252-5fc58f34aac1 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:17:02.819520 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ed423585-6395-45c0-ae8a-3cf3a186c9be admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.880728 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ed423585-6395-45c0-ae8a-3cf3a186c9be admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.959232 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ed423585-6395-45c0-ae8a-3cf3a186c9be admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.971584 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:02.990744 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ed423585-6395-45c0-ae8a-3cf3a186c9be admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1e88b4a0-fe96-459c-a8f3-3e43be9a4979,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:02.995525 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ed423585-6395-45c0-ae8a-3cf3a186c9be admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:02.997624 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ed423585-6395-45c0-ae8a-3cf3a186c9be admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:02.999558 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-ed423585-6395-45c0-ae8a-3cf3a186c9be admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/1e88b4a0-fe96-459c-a8f3-3e43be9a4979 HTTP/1.1" status: 204 len: 168 time: 0.3693879 May 24 11:17:03.270437 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-899963d6-8ae3-438c-ba49-a85d9a3552ff req-e645afe6-8a89-47fc-94d1-400fabf80128 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:03.272271 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-899963d6-8ae3-438c-ba49-a85d9a3552ff req-e645afe6-8a89-47fc-94d1-400fabf80128 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1272430 May 24 11:17:03.336946 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-899963d6-8ae3-438c-ba49-a85d9a3552ff req-0866f670-29d6-40d8-acc2-f06a0d422d24 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0567415 May 24 11:17:03.466281 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53410) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:03.485458 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-6081a293-3ddf-4e9c-b718-4db5af4f645f admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:03.502570 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-6081a293-3ddf-4e9c-b718-4db5af4f645f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:03.508076 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-6081a293-3ddf-4e9c-b718-4db5af4f645f admin admin] Usage tracker for resource:security_group_rule and tenant:1de4db3cf03941df8ee573008cf540c5 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:03.511792 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-6081a293-3ddf-4e9c-b718-4db5af4f645f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:03.518750 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-6081a293-3ddf-4e9c-b718-4db5af4f645f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:03.621818 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-6081a293-3ddf-4e9c-b718-4db5af4f645f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:03.627677 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-6081a293-3ddf-4e9c-b718-4db5af4f645f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:03.639937 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d967634a-43b7-44a0-b634-e3bfefac66b3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:03.682511 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6081a293-3ddf-4e9c-b718-4db5af4f645f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6b8a25af-6aa0-4e8d-b3a1-c73bdf24c623,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:03.689565 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-6081a293-3ddf-4e9c-b718-4db5af4f645f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:03.714114 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-6081a293-3ddf-4e9c-b718-4db5af4f645f admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=1de4db3cf03941df8ee573008cf540c5&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2465734 May 24 11:17:03.716595 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53420) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:03.844193 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f88bc2e3-a7ab-4a94-b05b-7360935e8938 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:03.893911 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f88bc2e3-a7ab-4a94-b05b-7360935e8938 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:03.974253 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3a4f7ee5-b021-4709-b431-fe6e2fca35b4 req-b901be9a-a5f2-4bb2-b71d-d00026adae19 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:03.979631 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3a4f7ee5-b021-4709-b431-fe6e2fca35b4 req-b901be9a-a5f2-4bb2-b71d-d00026adae19 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1631477 May 24 11:17:03.985027 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f88bc2e3-a7ab-4a94-b05b-7360935e8938 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:03.993218 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5e8f5c23-eca0-4874-8c89-ace3e8d8a672 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:04.008490 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f88bc2e3-a7ab-4a94-b05b-7360935e8938 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6b8a25af-6aa0-4e8d-b3a1-c73bdf24c623,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:04.011126 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f88bc2e3-a7ab-4a94-b05b-7360935e8938 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:04.012133 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f88bc2e3-a7ab-4a94-b05b-7360935e8938 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:04.013427 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-f88bc2e3-a7ab-4a94-b05b-7360935e8938 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/6b8a25af-6aa0-4e8d-b3a1-c73bdf24c623 HTTP/1.1" status: 204 len: 168 time: 0.2955153 May 24 11:17:04.063990 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3a4f7ee5-b021-4709-b431-fe6e2fca35b4 req-6bcc99d8-bfe7-484d-8e20-7f425fc9c5e2 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0656242 May 24 11:17:04.318012 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53440) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:04.486523 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e8cbc004-2c96-42ef-8886-b3899fc69fc1 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:04.514023 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e8cbc004-2c96-42ef-8886-b3899fc69fc1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:04.529187 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e8cbc004-2c96-42ef-8886-b3899fc69fc1 admin admin] Usage tracker for resource:security_group_rule and tenant:7cffc99115c4462e982420404a382c20 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:04.533026 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e8cbc004-2c96-42ef-8886-b3899fc69fc1 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:04.542192 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e8cbc004-2c96-42ef-8886-b3899fc69fc1 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:04.591380 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6cf8c41d-b3e6-48fa-a925-c0c092f27375 req-9f5352dc-9740-44f9-9a73-b3c5677a1d27 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:04.592939 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6cf8c41d-b3e6-48fa-a925-c0c092f27375 req-9f5352dc-9740-44f9-9a73-b3c5677a1d27 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1385834 May 24 11:17:04.730502 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6cf8c41d-b3e6-48fa-a925-c0c092f27375 req-bbf02e3b-0616-416c-846d-4816e19464f3 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1276870 May 24 11:17:04.765365 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e8cbc004-2c96-42ef-8886-b3899fc69fc1 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:04.769757 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e8cbc004-2c96-42ef-8886-b3899fc69fc1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:04.777375 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5b6ef236-f47e-480e-a2e1-1d98ceb7f12b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:04.821972 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e8cbc004-2c96-42ef-8886-b3899fc69fc1 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=7cffc99115c4462e982420404a382c20&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5005927 May 24 11:17:04.821972 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e8cbc004-2c96-42ef-8886-b3899fc69fc1 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=48325d60-0041-4c8d-8420-55c5e4f5ff6f,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:04.824725 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e8cbc004-2c96-42ef-8886-b3899fc69fc1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:04.825257 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53466) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:04.903346 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6592716c-238b-43a4-b44d-f06cbc8c459d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:04.930531 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6592716c-238b-43a4-b44d-f06cbc8c459d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:04.995348 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6592716c-238b-43a4-b44d-f06cbc8c459d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:05.001906 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:05.017825 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6592716c-238b-43a4-b44d-f06cbc8c459d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=48325d60-0041-4c8d-8420-55c5e4f5ff6f,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:05.021704 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6592716c-238b-43a4-b44d-f06cbc8c459d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:05.025417 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6592716c-238b-43a4-b44d-f06cbc8c459d admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:05.037963 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-6592716c-238b-43a4-b44d-f06cbc8c459d admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/48325d60-0041-4c8d-8420-55c5e4f5ff6f HTTP/1.1" status: 204 len: 168 time: 0.2003860 May 24 11:17:05.314500 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a09e83cb-b3a3-46bb-8736-e8a462f75187 req-25c735f2-2fa1-4beb-9760-9047f64de199 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:05.316380 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a09e83cb-b3a3-46bb-8736-e8a462f75187 req-25c735f2-2fa1-4beb-9760-9047f64de199 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1184781 May 24 11:17:05.370981 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a09e83cb-b3a3-46bb-8736-e8a462f75187 req-760114bf-f15b-4ac9-84a2-e480aafb71d2 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0430710 May 24 11:17:05.955759 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-537d5f79-ebff-4a0f-8633-a2b477622e7f req-cd10ab8d-48ee-4ccb-8f52-a63991043d8c tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:05.958911 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-537d5f79-ebff-4a0f-8633-a2b477622e7f req-cd10ab8d-48ee-4ccb-8f52-a63991043d8c tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1030362 May 24 11:17:06.029039 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-537d5f79-ebff-4a0f-8633-a2b477622e7f req-e99b4fb7-8201-4786-bcee-f9a57a037e68 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0611465 May 24 11:17:06.751756 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce4bf54c-2721-494d-9991-f114659a2921 req-d5b7d97f-09f7-4c8d-8b43-996195c8a767 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:06.757578 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53572) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:06.764720 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce4bf54c-2721-494d-9991-f114659a2921 req-d5b7d97f-09f7-4c8d-8b43-996195c8a767 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1951773 May 24 11:17:06.813936 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce4bf54c-2721-494d-9991-f114659a2921 req-ea6dd15f-b8c2-4634-9e7a-8030be4dd52d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0479474 May 24 11:17:06.922609 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0d546cf0-183f-4cfd-a318-ed1435ec5ab7 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:06.946033 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-0d546cf0-183f-4cfd-a318-ed1435ec5ab7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:06.951459 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-0d546cf0-183f-4cfd-a318-ed1435ec5ab7 admin admin] Usage tracker for resource:security_group_rule and tenant:0da3d06918854294bc3e81dafa9573a7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:06.956042 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-0d546cf0-183f-4cfd-a318-ed1435ec5ab7 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:06.964495 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-0d546cf0-183f-4cfd-a318-ed1435ec5ab7 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:07.103732 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0d546cf0-183f-4cfd-a318-ed1435ec5ab7 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:07.111507 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0d546cf0-183f-4cfd-a318-ed1435ec5ab7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:07.148016 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:07.157868 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-02b06d28-3af1-4dd9-97b7-030f692e397d req-4d6615b8-617e-4f1d-8c7d-e32d7d74f038 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:07.167167 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-02b06d28-3af1-4dd9-97b7-030f692e397d req-4d6615b8-617e-4f1d-8c7d-e32d7d74f038 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 945 time: 0.1079526 May 24 11:17:07.180059 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-0d546cf0-183f-4cfd-a318-ed1435ec5ab7 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=0da3d06918854294bc3e81dafa9573a7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4213912 May 24 11:17:07.182670 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0d546cf0-183f-4cfd-a318-ed1435ec5ab7 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d08428cb-07cd-4778-a2f3-fe4c9c5bd139,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:07.184052 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53592) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:07.191669 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-0d546cf0-183f-4cfd-a318-ed1435ec5ab7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:07.256604 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-02b06d28-3af1-4dd9-97b7-030f692e397d req-55bc96d5-cb36-4051-beef-bb64df06b1f5 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0882332 May 24 11:17:07.290214 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6a3ba289-b539-4e83-9aff-c067d69e4b4e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:07.299789 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e553688-721b-4327-bfc5-016118a57120 req-3440ae1f-2f30-409f-800f-1ee1530682e5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:07.301342 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0e553688-721b-4327-bfc5-016118a57120 req-3440ae1f-2f30-409f-800f-1ee1530682e5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1198752 May 24 11:17:07.319165 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6a3ba289-b539-4e83-9aff-c067d69e4b4e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:07.357043 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0e553688-721b-4327-bfc5-016118a57120 req-3b522200-61da-4472-95d0-de0b51a1930f tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0485191 May 24 11:17:07.396528 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6a3ba289-b539-4e83-9aff-c067d69e4b4e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:07.401782 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b51c199d-7438-4188-b99c-c2df2184b977 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:07.424517 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6a3ba289-b539-4e83-9aff-c067d69e4b4e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d08428cb-07cd-4778-a2f3-fe4c9c5bd139,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:07.429768 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6a3ba289-b539-4e83-9aff-c067d69e4b4e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:07.430984 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6a3ba289-b539-4e83-9aff-c067d69e4b4e admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:07.432911 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-6a3ba289-b539-4e83-9aff-c067d69e4b4e admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/d08428cb-07cd-4778-a2f3-fe4c9c5bd139 HTTP/1.1" status: 204 len: 168 time: 0.2479348 May 24 11:17:07.608409 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e663d1e-bdd0-4541-a557-15f312561431 req-b92353a4-b046-45ad-a78b-42b4b250a647 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:07.610005 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5e663d1e-bdd0-4541-a557-15f312561431 req-b92353a4-b046-45ad-a78b-42b4b250a647 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.0934653 May 24 11:17:07.619812 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 53610) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:07.625539 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-0e937ea5-873e-471c-9faa-c3136e08f444 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Request body: {'floatingip': {'port_id': 'eab44150-44f7-4194-9b76-436d421794e8'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:17:07.713209 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0e937ea5-873e-471c-9faa-c3136e08f444 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:08.007172 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53620) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:08.143707 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5b864378-33ac-4116-9533-f7fa2a0b873f admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:08.160473 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-5b864378-33ac-4116-9533-f7fa2a0b873f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:08.165237 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-5b864378-33ac-4116-9533-f7fa2a0b873f admin admin] Usage tracker for resource:security_group_rule and tenant:2dbf1511441b4b2997ec12c0022ddddb is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:08.168092 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-5b864378-33ac-4116-9533-f7fa2a0b873f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:08.173835 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-5b864378-33ac-4116-9533-f7fa2a0b873f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:08.274057 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5b864378-33ac-4116-9533-f7fa2a0b873f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:08.281298 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5b864378-33ac-4116-9533-f7fa2a0b873f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:08.303035 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-9eca99f1-ac8f-4b93-90d8-9208a157db23 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:08.353010 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b864378-33ac-4116-9533-f7fa2a0b873f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=865ebc27-37be-4464-a9e6-b5b816c19a9f,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:08.359223 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-5b864378-33ac-4116-9533-f7fa2a0b873f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:08.363511 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-5b864378-33ac-4116-9533-f7fa2a0b873f admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=2dbf1511441b4b2997ec12c0022ddddb&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3551767 May 24 11:17:08.366826 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53642) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:08.447066 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0246a50-f3bc-44a0-8b81-b4f4f6d67962 req-e0b36489-9c9a-4ce0-b8ba-a52da3c7210d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:08.454636 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e0246a50-f3bc-44a0-8b81-b4f4f6d67962 req-e0b36489-9c9a-4ce0-b8ba-a52da3c7210d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.4654448 May 24 11:17:08.515254 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e0246a50-f3bc-44a0-8b81-b4f4f6d67962 req-504235c2-67ba-40b6-8cfe-986994b8e132 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0596833 May 24 11:17:08.516995 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-263aca09-6cab-42d6-90f9-ab160d7977d1 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:08.560135 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a5908781-ef39-45a0-963c-42c130db1572 req-88b41242-1c11-4035-8e64-b42d187275cc tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:08.561339 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a5908781-ef39-45a0-963c-42c130db1572 req-88b41242-1c11-4035-8e64-b42d187275cc tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 945 time: 0.1704743 May 24 11:17:08.591318 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-263aca09-6cab-42d6-90f9-ab160d7977d1 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:08.652772 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a5908781-ef39-45a0-963c-42c130db1572 req-0ed73e61-e3f3-4fe9-9f86-33109ca06452 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0835166 May 24 11:17:08.683690 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-263aca09-6cab-42d6-90f9-ab160d7977d1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:08.689750 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-47e4eb83-47c3-440f-a48b-2b3036f76bbf admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:08.713108 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-263aca09-6cab-42d6-90f9-ab160d7977d1 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=865ebc27-37be-4464-a9e6-b5b816c19a9f,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:08.716039 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-263aca09-6cab-42d6-90f9-ab160d7977d1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:08.733934 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-263aca09-6cab-42d6-90f9-ab160d7977d1 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:08.737728 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-263aca09-6cab-42d6-90f9-ab160d7977d1 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/865ebc27-37be-4464-a9e6-b5b816c19a9f HTTP/1.1" status: 204 len: 168 time: 0.3699203 May 24 11:17:08.810294 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0e937ea5-873e-471c-9faa-c3136e08f444 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:08.816073 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0e937ea5-873e-471c-9faa-c3136e08f444 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:09.002464 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [None req-0e937ea5-873e-471c-9faa-c3136e08f444 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Floating IP 39b51eeb-5978-4a5b-8a19-eaa131930a0b associated. External IP: 172.27.0.86, port: eab44150-44f7-4194-9b76-436d421794e8. May 24 11:17:09.109978 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53662) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:09.127537 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f15036e6-45b0-41b8-a8a4-4ad211f15e23 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:09.142439 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-f15036e6-45b0-41b8-a8a4-4ad211f15e23 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:09.146743 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-f15036e6-45b0-41b8-a8a4-4ad211f15e23 admin admin] Usage tracker for resource:security_group_rule and tenant:00042fa9786e4f2d97f69b4b345ed6f2 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:09.149698 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-f15036e6-45b0-41b8-a8a4-4ad211f15e23 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:09.155632 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-f15036e6-45b0-41b8-a8a4-4ad211f15e23 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:09.333751 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f15036e6-45b0-41b8-a8a4-4ad211f15e23 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:09.339600 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f15036e6-45b0-41b8-a8a4-4ad211f15e23 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:09.349707 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:09.375451 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-52d06cda-bfd5-4188-969e-33dd7b7856a6 req-65d73c47-baa1-4f1f-b332-d85e20621406 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:09.376990 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-52d06cda-bfd5-4188-969e-33dd7b7856a6 req-65d73c47-baa1-4f1f-b332-d85e20621406 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.93 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 945 time: 0.1869879 May 24 11:17:09.402871 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-0e937ea5-873e-471c-9faa-c3136e08f444 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Router f76568b3-17ef-487b-beeb-3f592a58e138 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:17:09.454798 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-f15036e6-45b0-41b8-a8a4-4ad211f15e23 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=00042fa9786e4f2d97f69b4b345ed6f2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3437231 May 24 11:17:09.455613 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f15036e6-45b0-41b8-a8a4-4ad211f15e23 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=105e0853-f27b-4199-b7c9-628048c804d0,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:09.461460 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53672) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:09.466680 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f15036e6-45b0-41b8-a8a4-4ad211f15e23 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.118s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:09.599181 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-34fabc75-a008-4282-9769-ef1ddfe5b428 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:09.624580 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [req-52d06cda-bfd5-4188-969e-33dd7b7856a6 req-78b94553-a416-4043-91df-d8ddb1bdc4bb tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Deleting port 65acda30-2128-434e-a144-f18a69777754 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:17:09.624931 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-52d06cda-bfd5-4188-969e-33dd7b7856a6 req-78b94553-a416-4043-91df-d8ddb1bdc4bb tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:09.627888 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-0e937ea5-873e-471c-9faa-c3136e08f444 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:17:09.644915 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ea09b6d4-e286-499c-be53-941316c0df77 None None] Sync routers for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:17:09.661878 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ea09b6d4-e286-499c-be53-941316c0df77 None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:17:09.663635 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-ea09b6d4-e286-499c-be53-941316c0df77 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:16:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:09.677658 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0e937ea5-873e-471c-9faa-c3136e08f444 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:09.700369 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-34fabc75-a008-4282-9769-ef1ddfe5b428 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:09.844299 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-0e937ea5-873e-471c-9faa-c3136e08f444 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "PUT /v2.0/floatingips/39b51eeb-5978-4a5b-8a19-eaa131930a0b HTTP/1.1" status: 200 len: 991 time: 2.2230508 May 24 11:17:09.846673 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'tag': 'eab44150-44f7-4194-9b76-436d421794e8'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:17:09.896488 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8a2da2cd-5c78-4dc1-b334-e6568ff377c3 req-2781f2d9-d76e-4381-aa48-9d729934d88f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:09.898430 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8a2da2cd-5c78-4dc1-b334-e6568ff377c3 req-2781f2d9-d76e-4381-aa48-9d729934d88f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1729672 May 24 11:17:09.902976 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-ea09b6d4-e286-499c-be53-941316c0df77 None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:17:09.991992 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8a2da2cd-5c78-4dc1-b334-e6568ff377c3 req-e12f3a24-5819-496b-a879-c8b47f0a5183 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0844142 May 24 11:17:10.010434 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-52d06cda-bfd5-4188-969e-33dd7b7856a6 req-78b94553-a416-4043-91df-d8ddb1bdc4bb tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] For port 65acda30-2128-434e-a144-f18a69777754, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=65acda30-2128-434e-a144-f18a69777754,segment=NetworkSegment(b6a611a2-11cb-43b7-8194-22ac1ef4914b),segment_id=b6a611a2-11cb-43b7-8194-22ac1ef4914b)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:17:10.011026 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-52d06cda-bfd5-4188-969e-33dd7b7856a6 req-78b94553-a416-4043-91df-d8ddb1bdc4bb tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:10.176225 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [req-52d06cda-bfd5-4188-969e-33dd7b7856a6 req-78b94553-a416-4043-91df-d8ddb1bdc4bb tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Calling delete_port for 65acda30-2128-434e-a144-f18a69777754 owned by compute:nova {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:17:10.289594 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'tag': 'eab44150-44f7-4194-9b76-436d421794e8', 'status': 'completed', 'code': 200} May 24 11:17:10.356033 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6dd57274-f46f-425e-81c2-5a0212e4b26a req-29175753-32ad-43e5-ac6d-e73731b7550e tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:10.362956 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6dd57274-f46f-425e-81c2-5a0212e4b26a req-29175753-32ad-43e5-ac6d-e73731b7550e tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 945 time: 0.4790015 May 24 11:17:10.365940 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-34fabc75-a008-4282-9769-ef1ddfe5b428 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:10.380419 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-8c3be379-33bc-4fd9-ae64-bc7d30f1fa4f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:10.429895 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-34fabc75-a008-4282-9769-ef1ddfe5b428 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=105e0853-f27b-4199-b7c9-628048c804d0,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:10.432265 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-34fabc75-a008-4282-9769-ef1ddfe5b428 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:10.432733 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-ea09b6d4-e286-499c-be53-941316c0df77 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '39b51eeb-5978-4a5b-8a19-eaa131930a0b', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'floating_ip_address': '172.27.0.86', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'port_id': 'eab44150-44f7-4194-9b76-436d421794e8', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 119, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'mac_address': 'fa:16:3e:b2:cf:de', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:16:23Z', 'updated_at': '2021-05-24T11:17:08Z', 'revision_number': 1, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:17Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:10.439287 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ea09b6d4-e286-499c-be53-941316c0df77 None None] Checking router: f76568b3-17ef-487b-beeb-3f592a58e138 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:17:10.443919 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-34fabc75-a008-4282-9769-ef1ddfe5b428 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:10.447910 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-34fabc75-a008-4282-9769-ef1ddfe5b428 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/105e0853-f27b-4199-b7c9-628048c804d0 HTTP/1.1" status: 204 len: 168 time: 0.9851205 May 24 11:17:10.464329 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7995b6f-eb1a-4b68-a583-5e49790faf9d req-9d472bf8-a065-446f-a912-6a2fd5aaf21b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:10.464329 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a7995b6f-eb1a-4b68-a583-5e49790faf9d req-9d472bf8-a065-446f-a912-6a2fd5aaf21b service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=b303f83a84794e5ba6a1bab9c76b429c&device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 1099 time: 0.1462827 May 24 11:17:10.477712 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ea09b6d4-e286-499c-be53-941316c0df77 None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:17:10.478201 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6dd57274-f46f-425e-81c2-5a0212e4b26a req-505f10a8-8a87-498e-9678-ca19d3f456e7 tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/security-groups?id=058e967f-9655-4f0a-9bd1-d825ec089aa1&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1008892 May 24 11:17:10.478602 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-ea09b6d4-e286-499c-be53-941316c0df77 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:16:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:10.648528 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-52d06cda-bfd5-4188-969e-33dd7b7856a6 req-78b94553-a416-4043-91df-d8ddb1bdc4bb tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:10.663786 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['65acda30-2128-434e-a144-f18a69777754'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:17:10.670699 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:10.736822 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-ea09b6d4-e286-499c-be53-941316c0df77 None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:17:10.822189 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-78b94553-a416-4043-91df-d8ddb1bdc4bb tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Pushing event deleted for resources: {'Port': ['ID=65acda30-2128-434e-a144-f18a69777754,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:10.824982 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-78b94553-a416-4043-91df-d8ddb1bdc4bb tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.154s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:10.847098 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7995b6f-eb1a-4b68-a583-5e49790faf9d req-133ea20b-99aa-4908-a511-b8fc674ee3e3 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:10.849414 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a7995b6f-eb1a-4b68-a583-5e49790faf9d req-133ea20b-99aa-4908-a511-b8fc674ee3e3 service neutron] 10.222.0.93 "GET /v2.0/networks?id=1d92641a-d8f9-4ed6-9abd-f464bb26811d HTTP/1.1" status: 200 len: 912 time: 0.3696830 May 24 11:17:10.953671 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [req-52d06cda-bfd5-4188-969e-33dd7b7856a6 req-78b94553-a416-4043-91df-d8ddb1bdc4bb tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] No DVR routers for this DVR port 65acda30-2128-434e-a144-f18a69777754 on host hv-cinder-78409 {{(pid=91820) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:17:10.957603 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-52d06cda-bfd5-4188-969e-33dd7b7856a6 req-78b94553-a416-4043-91df-d8ddb1bdc4bb tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:10.962732 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '82ca8978-fbb3-4792-8bfd-f8b2e565b925', 'name': 'network-vif-deleted', 'tag': '65acda30-2128-434e-a144-f18a69777754'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:17:10.982157 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-52d06cda-bfd5-4188-969e-33dd7b7856a6 req-78b94553-a416-4043-91df-d8ddb1bdc4bb tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.93 "DELETE /v2.0/ports/65acda30-2128-434e-a144-f18a69777754 HTTP/1.1" status: 204 len: 173 time: 1.5875268 May 24 11:17:11.016295 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a7995b6f-eb1a-4b68-a583-5e49790faf9d req-246265fd-16fd-4754-abaa-5cc6a8a50ab9 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=eab44150-44f7-4194-9b76-436d421794e8 HTTP/1.1" status: 200 len: 999 time: 0.1472297 May 24 11:17:11.031719 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '82ca8978-fbb3-4792-8bfd-f8b2e565b925', 'name': 'network-vif-deleted', 'tag': '65acda30-2128-434e-a144-f18a69777754', 'status': 'completed', 'code': 200} May 24 11:17:11.150545 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7995b6f-eb1a-4b68-a583-5e49790faf9d req-4bdd6d54-bde2-47d6-87e4-4f3dee8b94d8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:11.152298 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a7995b6f-eb1a-4b68-a583-5e49790faf9d req-4bdd6d54-bde2-47d6-87e4-4f3dee8b94d8 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=2fc4be2e-875c-4d10-b857-a48f90c53cf2 HTTP/1.1" status: 200 len: 850 time: 0.1273859 May 24 11:17:11.255842 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7995b6f-eb1a-4b68-a583-5e49790faf9d req-6cc1273b-a2ec-4776-a79e-2b67ef0014d1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:11.262927 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a7995b6f-eb1a-4b68-a583-5e49790faf9d req-6cc1273b-a2ec-4776-a79e-2b67ef0014d1 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=1d92641a-d8f9-4ed6-9abd-f464bb26811d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0948682 May 24 11:17:11.319349 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e60b929-7ca7-4a64-9356-965b9b65d056 req-c912bc18-666c-4787-9115-cb5fb1b6ac0c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:11.320890 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5e60b929-7ca7-4a64-9356-965b9b65d056 req-c912bc18-666c-4787-9115-cb5fb1b6ac0c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1595430 May 24 11:17:11.430463 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5e60b929-7ca7-4a64-9356-965b9b65d056 req-0e033fac-ff85-47b7-9138-af887839631c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0986576 May 24 11:17:11.573195 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-ea09b6d4-e286-499c-be53-941316c0df77 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '39b51eeb-5978-4a5b-8a19-eaa131930a0b', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'floating_ip_address': '172.27.0.86', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'port_id': 'eab44150-44f7-4194-9b76-436d421794e8', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 119, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'mac_address': 'fa:16:3e:b2:cf:de', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:16:23Z', 'updated_at': '2021-05-24T11:17:08Z', 'revision_number': 1, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:17Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:11.585454 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ea09b6d4-e286-499c-be53-941316c0df77 None None] The sync data for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi is: [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '39b51eeb-5978-4a5b-8a19-eaa131930a0b', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'floating_ip_address': '172.27.0.86', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'port_id': 'eab44150-44f7-4194-9b76-436d421794e8', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 119, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'mac_address': 'fa:16:3e:b2:cf:de', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:16:23Z', 'updated_at': '2021-05-24T11:17:08Z', 'revision_number': 1, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:17Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:17:11.686681 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a7995b6f-eb1a-4b68-a583-5e49790faf9d req-b177da72-a2c7-46f8-9144-081adfbc87ef service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4180899 May 24 11:17:11.824161 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-94b1b470-7456-4df9-bfec-4f974ab384ca req-30d9d5dc-a83c-4e23-b351-9d9efeb3e9ec tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Synchronizing usage tracker for tenant:e65fade283614b028730cfea0f58bcad on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:17:11.861605 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-94b1b470-7456-4df9-bfec-4f974ab384ca req-30d9d5dc-a83c-4e23-b351-9d9efeb3e9ec tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Unset dirty status for tenant:e65fade283614b028730cfea0f58bcad on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:17:11.866416 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a7995b6f-eb1a-4b68-a583-5e49790faf9d req-ae264bde-9730-4387-a755-7c1a2708227f service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1709497 May 24 11:17:11.871618 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-94b1b470-7456-4df9-bfec-4f974ab384ca req-30d9d5dc-a83c-4e23-b351-9d9efeb3e9ec tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] 10.222.0.18 "GET /v2.0/ports?device_id=82ca8978-fbb3-4792-8bfd-f8b2e565b925 HTTP/1.1" status: 200 len: 210 time: 0.1638708 May 24 11:17:11.977050 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-08b64ce1-8e75-440c-b8b5-0be4e4d3208d None None] New status for floating IP 39b51eeb-5978-4a5b-8a19-eaa131930a0b: ACTIVE {{(pid=91821) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:17:12.640197 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1ef04141-787b-4485-83a3-bdcbc580d762 req-22be50c6-f22c-452e-9be4-0d6baab883f0 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:12.641729 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1ef04141-787b-4485-83a3-bdcbc580d762 req-22be50c6-f22c-452e-9be4-0d6baab883f0 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0993557 May 24 11:17:12.688759 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1ef04141-787b-4485-83a3-bdcbc580d762 req-12cebe6a-f8cf-40c7-9472-d7b51822c680 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0393817 May 24 11:17:13.630538 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 65acda30-2128-434e-a144-f18a69777754 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:17:13.673184 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 65acda30-2128-434e-a144-f18a69777754 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:17:13.673535 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 65acda30-2128-434e-a144-f18a69777754 not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:17:13.913164 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-58c94d12-b364-48d5-8c9c-0b229a146272 req-c112e27d-1750-4b74-a7a9-678b40fc818b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:13.914755 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-58c94d12-b364-48d5-8c9c-0b229a146272 req-c112e27d-1750-4b74-a7a9-678b40fc818b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1081240 May 24 11:17:13.970042 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-58c94d12-b364-48d5-8c9c-0b229a146272 req-dd54c5d1-4991-4896-ab22-605c37358ca8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0487540 May 24 11:17:14.328863 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9d4ae415-6e5b-4515-aedb-5ecb3205a9e7 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:14.330629 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9d4ae415-6e5b-4515-aedb-5ecb3205a9e7 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/networks?id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add HTTP/1.1" status: 200 len: 814 time: 0.1535561 May 24 11:17:14.345262 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-159121fd-ec81-4f33-9d40-5d69faad618c tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/quotas/003eb49c73ae4e6a962cbfb425b3c1eb HTTP/1.1" status: 200 len: 375 time: 0.0090714 May 24 11:17:14.425687 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bb2b6c6-fad7-4f27-bd77-a095b0741842 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&fields=id HTTP/1.1" status: 200 len: 255 time: 0.0751054 May 24 11:17:14.571962 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-3ff8899a-2a48-4340-9b65-3f356521c02f service neutron] 10.222.0.18 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1389074 May 24 11:17:14.725120 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-adbed68b-25f3-47fe-a0c5-5a0a7aca0ecc service neutron] 10.222.0.18 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1512222 May 24 11:17:14.740647 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53886) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:14.835143 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_v2 [None req-1ee2fcec-d7cc-4743-a98b-9dd4ac7b87ac admin admin] Deleting subnet 20bb60b1-11d7-4cba-b636-3bcdb98876e9 {{(pid=91820) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:17:14.838601 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1ee2fcec-d7cc-4743-a98b-9dd4ac7b87ac admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:14.911722 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1ee2fcec-d7cc-4743-a98b-9dd4ac7b87ac admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534861947'] for subnet, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:14.979896 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1ee2fcec-d7cc-4743-a98b-9dd4ac7b87ac admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:15.103347 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e1b3ad29-afc0-48cd-a730-a311ff61f2a1 req-69b71beb-d662-49a9-9fe6-69cc4a9e7425 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 210 time: 0.0918586 May 24 11:17:15.405134 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1dededae-b006-41e6-a6b4-158836ea6a13 req-0738e959-85ac-410b-a485-5c77bf1d85a5 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:15.406645 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1dededae-b006-41e6-a6b4-158836ea6a13 req-0738e959-85ac-410b-a485-5c77bf1d85a5 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1796012 May 24 11:17:15.435273 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1ee2fcec-d7cc-4743-a98b-9dd4ac7b87ac admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534788488', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535456750'] for subnet, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:15.443316 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:15.484926 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1ee2fcec-d7cc-4743-a98b-9dd4ac7b87ac admin admin] Pushing event deleted for resources: {'Subnet': ['ID=20bb60b1-11d7-4cba-b636-3bcdb98876e9,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:15.487193 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1ee2fcec-d7cc-4743-a98b-9dd4ac7b87ac admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:15.489821 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1ee2fcec-d7cc-4743-a98b-9dd4ac7b87ac admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:15.503344 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1dededae-b006-41e6-a6b4-158836ea6a13 req-44163bc3-6a9b-4b43-a825-acffe3ac941d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0890856 May 24 11:17:15.505479 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-1ee2fcec-d7cc-4743-a98b-9dd4ac7b87ac admin admin] 10.222.0.18 "DELETE /v2.0/subnets/20bb60b1-11d7-4cba-b636-3bcdb98876e9 HTTP/1.1" status: 204 len: 168 time: 0.7627113 May 24 11:17:15.507894 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53914) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:15.612538 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:15.741757 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:15.755429 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['29272b37-7e2e-4e92-9978-bb32dde1813d']}} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:15.767756 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'b6a611a2-11cb-43b7-8194-22ac1ef4914b') {'for_net_delete': True} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:15.768051 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'b6a611a2-11cb-43b7-8194-22ac1ef4914b') {} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:15.783403 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534861936'] for segment, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:15.804722 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:15.812157 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] Releasing vlan 543 on physical network physnet1 to pool {{(pid=91820) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:17:15.812699 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535262766'] for segment, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: Traceback (most recent call last): May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: timer() May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: cb(*args, **kw) May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: return func(*args, **kwargs) May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: self._notify() May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: self.callback(batched_events) May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: event.method(event) May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: return f(self, *a, **k) May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._get(url).json() May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: return self.request(url, 'GET', **kwargs) May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:17:15.827315 dv-cinder-784092-9-iscsi neutron-server[91820]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:17:15.912068 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:15.919717 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:15.922325 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-49ba4879-042e-48be-ba5e-ff35aad9498c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:15.944821 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] Pushing event deleted for resources: {'Network': ['ID=29272b37-7e2e-4e92-9978-bb32dde1813d,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:15.947434 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:15.956974 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-61c2393b-6ba2-4bb7-905a-26ae7a00b278 admin admin] 10.222.0.18 "DELETE /v2.0/networks/29272b37-7e2e-4e92-9978-bb32dde1813d HTTP/1.1" status: 204 len: 168 time: 0.4452026 May 24 11:17:16.157129 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53926) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:16.210887 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-6af51e04-dbd8-4f19-af86-5b3b8fa7d59d admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=e65fade283614b028730cfea0f58bcad&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0522716 May 24 11:17:16.213897 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 53930) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:16.290304 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6552838a-fd9d-46e1-82df-bb1fc9b04aa9 req-24b48021-96c6-435b-bc26-f6f164d64b2f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 210 time: 0.0615230 May 24 11:17:16.327514 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-96065557-498a-401c-85df-1707e6a1474b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:16.356439 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-96065557-498a-401c-85df-1707e6a1474b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:16.412753 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-96065557-498a-401c-85df-1707e6a1474b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:16.416560 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-263aca09-6cab-42d6-90f9-ab160d7977d1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:16.429911 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-96065557-498a-401c-85df-1707e6a1474b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=058e967f-9655-4f0a-9bd1-d825ec089aa1,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:16.433170 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-96065557-498a-401c-85df-1707e6a1474b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:16.445562 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-96065557-498a-401c-85df-1707e6a1474b admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:16.446846 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-96065557-498a-401c-85df-1707e6a1474b admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/058e967f-9655-4f0a-9bd1-d825ec089aa1 HTTP/1.1" status: 204 len: 168 time: 0.2317989 May 24 11:17:16.800306 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-519b6332-57c9-42fa-a97d-f656150aa645 req-b59ffe44-0342-4db7-b5b8-be2e67958121 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:16.802033 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-519b6332-57c9-42fa-a97d-f656150aa645 req-b59ffe44-0342-4db7-b5b8-be2e67958121 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1782534 May 24 11:17:16.857856 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-519b6332-57c9-42fa-a97d-f656150aa645 req-d749b8b2-d7f7-4182-9362-fdd228de50fc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0495384 May 24 11:17:17.445558 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a2088471-ed3f-4c58-86a5-5ea766dad00c req-24447b48-cba6-46bd-aa9b-7926044ea075 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 210 time: 0.0436306 May 24 11:17:18.086685 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e124155d-4856-44ad-aa9f-be86f21fde53 req-2c2e678e-e986-491c-a03b-aa25418463af tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:18.088359 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e124155d-4856-44ad-aa9f-be86f21fde53 req-2c2e678e-e986-491c-a03b-aa25418463af tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1074934 May 24 11:17:18.141723 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e124155d-4856-44ad-aa9f-be86f21fde53 req-7af38072-abf6-41ff-a625-ee39f292270f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0461700 May 24 11:17:18.507124 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-6c5c96ee-56ce-4eef-a825-bcc25840eae9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:18.509297 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-6c5c96ee-56ce-4eef-a825-bcc25840eae9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "GET /v2.0/networks?id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add HTTP/1.1" status: 200 len: 814 time: 0.1938021 May 24 11:17:18.527379 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Request body: {'port': {'device_id': 'd895da25-06b9-484c-8192-a6e4df06a856', 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add', 'admin_state_up': True, 'tenant_id': '003eb49c73ae4e6a962cbfb425b3c1eb'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:17:18.783051 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bf038be6-ba5e-4fac-bfe7-a556cdfe485f req-0f9d1e38-8708-45bc-ad5c-2af709b4c555 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 210 time: 0.1232970 May 24 11:17:18.844454 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:18.872184 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attempting to reserve 1 items for resource port. Total usage: 1; quota limit: 500; headroom:499 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:18.885458 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Made reservation on behalf of 003eb49c73ae4e6a962cbfb425b3c1eb for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:17:18.890436 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:19.074335 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Allocated IP 172.20.0.4 (bb3e0f6d-2046-4828-8f43-5f99ba5f3add/1107b450-6c27-4f13-8b6e-abdbbbb1ea0f/159fe3b9-426c-472e-9870-2ead1792535a) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:17:19.396033 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:19.410666 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45af5f27-6d3e-490b-bb1a-93fd673306d3 req-2d7eed83-1e2d-4e44-b191-f542f01a186c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:19.413257 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-45af5f27-6d3e-490b-bb1a-93fd673306d3 req-2d7eed83-1e2d-4e44-b191-f542f01a186c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1130109 May 24 11:17:19.436252 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:19.442553 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:19.450802 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['45911d6f-9595-40c5-8b29-75259c824bfe'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:17:19.480960 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-45af5f27-6d3e-490b-bb1a-93fd673306d3 req-de0c348a-f3f8-4523-a3f9-3a05b4ad3bd4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0587220 May 24 11:17:19.610420 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Network bb3e0f6d-2046-4828-8f43-5f99ba5f3add is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:17:19.614425 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:17:19.616399 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Pushing event updated for resources: {'Port': ['ID=159fe3b9-426c-472e-9870-2ead1792535a,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:19.629175 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:17:19.694914 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.252s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:19.716361 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:19.717046 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:19.756104 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.2336245 May 24 11:17:19.772042 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Request body: {'port': {'device_id': 'd895da25-06b9-484c-8192-a6e4df06a856', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:17:19.828350 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f486678c-6db7-4586-af01-ee93cd81451f req-207687ed-1059-4b60-8501-98fc73c17aee service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1083035 May 24 11:17:19.929019 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:20.170375 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a624674d-4aed-4193-bc1f-e34bc7925cc1 req-d1e9f861-76a8-4d8e-ae60-2712be01310a tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:20.172316 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a624674d-4aed-4193-bc1f-e34bc7925cc1 req-d1e9f861-76a8-4d8e-ae60-2712be01310a tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 931 time: 0.2053928 May 24 11:17:20.189389 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f486678c-6db7-4586-af01-ee93cd81451f req-dfd233b5-8a74-41b9-a544-7c2e22cc1374 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:20.192954 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f486678c-6db7-4586-af01-ee93cd81451f req-dfd233b5-8a74-41b9-a544-7c2e22cc1374 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 1098 time: 0.1530221 May 24 11:17:20.361431 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a624674d-4aed-4193-bc1f-e34bc7925cc1 req-4126ca53-46b7-45d6-bebd-6911a50d5c3d tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1672816 May 24 11:17:20.370172 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 159fe3b9-426c-472e-9870-2ead1792535a {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:17:20.385151 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:20.397267 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Transition to ACTIVE for port object 159fe3b9-426c-472e-9870-2ead1792535a will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:17:20.406969 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:20.420053 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-6de83924-4c41-457d-b19f-664e4c734ef4 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:20.422526 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f486678c-6db7-4586-af01-ee93cd81451f req-585dcf0e-41c5-4f99-a1e2-1a65fc85cd54 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:20.424587 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f486678c-6db7-4586-af01-ee93cd81451f req-585dcf0e-41c5-4f99-a1e2-1a65fc85cd54 service neutron] 10.222.0.93 "GET /v2.0/networks?id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add HTTP/1.1" status: 200 len: 915 time: 0.1814246 May 24 11:17:20.573433 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f486678c-6db7-4586-af01-ee93cd81451f req-a337ba46-5819-402d-a1f1-df2239f39120 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=45911d6f-9595-40c5-8b29-75259c824bfe HTTP/1.1" status: 200 len: 217 time: 0.1340506 May 24 11:17:20.618410 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Pushing event updated for resources: {'Port': ['ID=159fe3b9-426c-472e-9870-2ead1792535a,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:20.623399 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.203s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:20.664325 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f486678c-6db7-4586-af01-ee93cd81451f req-c21039dc-d447-40a4-9163-f0cb5cd24103 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:20.668898 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f486678c-6db7-4586-af01-ee93cd81451f req-c21039dc-d447-40a4-9163-f0cb5cd24103 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=1107b450-6c27-4f13-8b6e-abdbbbb1ea0f HTTP/1.1" status: 200 len: 854 time: 0.0914733 May 24 11:17:20.746712 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f486678c-6db7-4586-af01-ee93cd81451f req-f942223f-429f-4e80-aa54-339aff783ed6 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0699816 May 24 11:17:20.833032 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Attempting to bind port 159fe3b9-426c-472e-9870-2ead1792535a on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:17:20.833393 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Attempting to bind port 159fe3b9-426c-472e-9870-2ead1792535a by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:17:20.833725 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Attempting to bind port 159fe3b9-426c-472e-9870-2ead1792535a on network bb3e0f6d-2046-4828-8f43-5f99ba5f3add {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:17:20.842910 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Port 159fe3b9-426c-472e-9870-2ead1792535a on network bb3e0f6d-2046-4828-8f43-5f99ba5f3add not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:17:20.843250 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Attempting to bind port 159fe3b9-426c-472e-9870-2ead1792535a on network bb3e0f6d-2046-4828-8f43-5f99ba5f3add {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:17:20.854897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 16, 51), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:17:20.854897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Checking segment: {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:17:20.854897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Bound using segment: {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:17:20.854897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Bound port: 159fe3b9-426c-472e-9870-2ead1792535a, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:17:20.854897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:20.931117 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1bf5893-2f19-4816-9309-2b09f9d7437e req-2f2387b1-c1a9-4bc0-926b-630015f67959 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:20.931820 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d1bf5893-2f19-4816-9309-2b09f9d7437e req-2f2387b1-c1a9-4bc0-926b-630015f67959 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.2437584 May 24 11:17:20.965106 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f486678c-6db7-4586-af01-ee93cd81451f req-5986f4ed-8bfa-4287-9d60-3565bcc84110 service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2074866 May 24 11:17:21.028508 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d1bf5893-2f19-4816-9309-2b09f9d7437e req-2ae18102-c51c-4a0b-b046-e8b243d36274 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0953071 May 24 11:17:21.047021 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] For port 159fe3b9-426c-472e-9870-2ead1792535a, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:17:21.055678 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] For port 159fe3b9-426c-472e-9870-2ead1792535a, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=159fe3b9-426c-472e-9870-2ead1792535a,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:17:21.061445 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:21.073210 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:21.119178 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:21.121184 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:21.123097 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] 10.222.0.93 "PUT /v2.0/ports/159fe3b9-426c-472e-9870-2ead1792535a HTTP/1.1" status: 200 len: 1093 time: 1.3582032 May 24 11:17:21.133854 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'd895da25-06b9-484c-8192-a6e4df06a856', 'tag': '159fe3b9-426c-472e-9870-2ead1792535a'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:17:21.185217 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Pushing event updated for resources: {'Port': ['ID=159fe3b9-426c-472e-9870-2ead1792535a,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:21.194695 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.121s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:21.221543 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'd895da25-06b9-484c-8192-a6e4df06a856', 'tag': '159fe3b9-426c-472e-9870-2ead1792535a', 'status': 'completed', 'code': 200} May 24 11:17:21.561067 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1d794fc-d450-43e0-9be8-eef6cb716eb7 req-4e5477ec-38a6-42fd-a136-949453082891 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:21.561815 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-e1d794fc-d450-43e0-9be8-eef6cb716eb7 req-4e5477ec-38a6-42fd-a136-949453082891 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Synchronizing usage tracker for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:17:21.579315 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-e1d794fc-d450-43e0-9be8-eef6cb716eb7 req-4e5477ec-38a6-42fd-a136-949453082891 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Unset dirty status for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:17:21.580734 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e1d794fc-d450-43e0-9be8-eef6cb716eb7 req-4e5477ec-38a6-42fd-a136-949453082891 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 943 time: 0.1042991 May 24 11:17:21.636403 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e1d794fc-d450-43e0-9be8-eef6cb716eb7 req-c047fc17-0c34-440a-8d4a-8bdb37b7368e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0494297 May 24 11:17:21.798322 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 54110) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:21.929232 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7d2151b9-336c-49b2-8966-1ca2bd421f8e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:21.944825 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-7d2151b9-336c-49b2-8966-1ca2bd421f8e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:21.949209 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-7d2151b9-336c-49b2-8966-1ca2bd421f8e admin admin] Usage tracker for resource:security_group_rule and tenant:b893d7eafa744d028f2ac8d24b135c2d is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:21.952109 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-7d2151b9-336c-49b2-8966-1ca2bd421f8e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:21.957962 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-7d2151b9-336c-49b2-8966-1ca2bd421f8e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:21.974799 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:22.046757 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7d2151b9-336c-49b2-8966-1ca2bd421f8e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:22.052237 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7d2151b9-336c-49b2-8966-1ca2bd421f8e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:22.062818 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-0d546cf0-183f-4cfd-a318-ed1435ec5ab7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:22.110119 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-7d2151b9-336c-49b2-8966-1ca2bd421f8e admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=b893d7eafa744d028f2ac8d24b135c2d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3083901 May 24 11:17:22.110119 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7d2151b9-336c-49b2-8966-1ca2bd421f8e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1d6a7845-b68c-4178-aa39-577d4b01f4d4,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:22.111092 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54120) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:22.113334 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-7d2151b9-336c-49b2-8966-1ca2bd421f8e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:22.181443 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-04a73591-899b-47ad-ad9e-a078a5ac2f8b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:22.208153 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-04a73591-899b-47ad-ad9e-a078a5ac2f8b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:22.226080 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e8f1f6d1-291e-4c5e-bd58-a7901a89236f req-ef8c81be-52fa-409e-86e7-9bf1adff2529 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:22.227055 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e8f1f6d1-291e-4c5e-bd58-a7901a89236f req-ef8c81be-52fa-409e-86e7-9bf1adff2529 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0864115 May 24 11:17:22.263027 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-04a73591-899b-47ad-ad9e-a078a5ac2f8b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:22.268456 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ae904b1c-0eab-49e5-a0cc-3bdd0910104e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:22.275884 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e8f1f6d1-291e-4c5e-bd58-a7901a89236f req-a307923a-f3ab-4f86-9378-9645e95e7dee tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0430617 May 24 11:17:22.280363 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-04a73591-899b-47ad-ad9e-a078a5ac2f8b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1d6a7845-b68c-4178-aa39-577d4b01f4d4,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:22.283212 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-04a73591-899b-47ad-ad9e-a078a5ac2f8b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:22.285308 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-04a73591-899b-47ad-ad9e-a078a5ac2f8b admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:22.297696 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-04a73591-899b-47ad-ad9e-a078a5ac2f8b admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/1d6a7845-b68c-4178-aa39-577d4b01f4d4 HTTP/1.1" status: 204 len: 168 time: 0.1783135 May 24 11:17:22.588112 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 54130) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:22.606394 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8b73facb-c480-46c4-b1e2-90eecf3c5210 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:22.621336 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-8b73facb-c480-46c4-b1e2-90eecf3c5210 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:22.625038 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-8b73facb-c480-46c4-b1e2-90eecf3c5210 admin admin] Usage tracker for resource:security_group_rule and tenant:cfa88a1c7e1b4a5294094ba6a0830824 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:22.627799 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-8b73facb-c480-46c4-b1e2-90eecf3c5210 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:22.633744 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-8b73facb-c480-46c4-b1e2-90eecf3c5210 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:22.726870 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8b73facb-c480-46c4-b1e2-90eecf3c5210 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:22.732946 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8b73facb-c480-46c4-b1e2-90eecf3c5210 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:22.742330 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-afb46b31-38b8-4d62-9b88-2d0cb3479b15 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:22.832577 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-8b73facb-c480-46c4-b1e2-90eecf3c5210 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=cfa88a1c7e1b4a5294094ba6a0830824&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2433257 May 24 11:17:22.834685 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8b73facb-c480-46c4-b1e2-90eecf3c5210 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=885eb1f8-ecc2-4bbb-a5d3-0664a12b9989,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:22.835362 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54138) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:22.838103 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8b73facb-c480-46c4-b1e2-90eecf3c5210 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:22.887286 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7a71d33f-de80-4097-ad1e-619ebaa5ad93 req-e4146519-865d-4161-8bd1-d6597a508238 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:22.889296 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7a71d33f-de80-4097-ad1e-619ebaa5ad93 req-e4146519-865d-4161-8bd1-d6597a508238 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 943 time: 0.1361322 May 24 11:17:22.909529 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f7617af6-6707-41a5-bf20-7f7e58a8d90f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:22.941235 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f7617af6-6707-41a5-bf20-7f7e58a8d90f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:22.945113 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7a71d33f-de80-4097-ad1e-619ebaa5ad93 req-beeee39e-fc33-4a9a-84fa-b197b374c883 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0502312 May 24 11:17:23.005557 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f7617af6-6707-41a5-bf20-7f7e58a8d90f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:23.029540 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-9a872824-8cb2-4bf1-8042-f88ec51bbeda admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:23.096921 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f7617af6-6707-41a5-bf20-7f7e58a8d90f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=885eb1f8-ecc2-4bbb-a5d3-0664a12b9989,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:23.102936 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f7617af6-6707-41a5-bf20-7f7e58a8d90f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:23.107495 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f7617af6-6707-41a5-bf20-7f7e58a8d90f admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:23.108630 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-f7617af6-6707-41a5-bf20-7f7e58a8d90f admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/885eb1f8-ecc2-4bbb-a5d3-0664a12b9989 HTTP/1.1" status: 204 len: 168 time: 0.2722862 May 24 11:17:23.175590 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f486678c-6db7-4586-af01-ee93cd81451f req-f7820145-bc02-42ea-ade2-6bf17ec5afd8 service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1720593 May 24 11:17:23.412403 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-66672bc9-d1e3-4019-a90d-aad1506d0b0e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:23.478082 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf41bbdf-3cf2-42ae-b017-b96b1bff32ae req-b657f250-62d6-437a-b9a7-72b8a6d7235b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:23.480045 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bf41bbdf-3cf2-42ae-b017-b96b1bff32ae req-b657f250-62d6-437a-b9a7-72b8a6d7235b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0880239 May 24 11:17:23.549081 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bf41bbdf-3cf2-42ae-b017-b96b1bff32ae req-8d80d163-4032-476f-80f4-f1b17f6094ed tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0616221 May 24 11:17:24.250338 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07e263b0-b6bf-4039-b0fa-112faa4a84c8 req-7158e047-ab7f-4de8-a9c0-73ec1f871932 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:24.252476 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-07e263b0-b6bf-4039-b0fa-112faa4a84c8 req-7158e047-ab7f-4de8-a9c0-73ec1f871932 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 943 time: 0.1777055 May 24 11:17:24.326141 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e5d29ab-864b-42dc-b038-8d4be52f88f2 req-dd5bc24c-413b-44a9-8b78-9e7b7b63dadc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:24.327687 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5e5d29ab-864b-42dc-b038-8d4be52f88f2 req-dd5bc24c-413b-44a9-8b78-9e7b7b63dadc service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 1096 time: 0.1246059 May 24 11:17:24.334706 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-07e263b0-b6bf-4039-b0fa-112faa4a84c8 req-348a0bae-b7ac-4258-8b16-c1858009bb9b tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0754092 May 24 11:17:24.481044 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e5d29ab-864b-42dc-b038-8d4be52f88f2 req-1ca11f3d-36ef-4ed6-a4ac-7fb3bb7ce55b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:24.482515 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5e5d29ab-864b-42dc-b038-8d4be52f88f2 req-1ca11f3d-36ef-4ed6-a4ac-7fb3bb7ce55b service neutron] 10.222.0.93 "GET /v2.0/networks?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&shared=False HTTP/1.1" status: 200 len: 915 time: 0.1466501 May 24 11:17:24.710581 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e5d29ab-864b-42dc-b038-8d4be52f88f2 req-77a4df76-d5ab-44b7-a391-09640744ac31 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:24.712221 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5e5d29ab-864b-42dc-b038-8d4be52f88f2 req-77a4df76-d5ab-44b7-a391-09640744ac31 service neutron] 10.222.0.93 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.2202864 May 24 11:17:24.884072 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-02056502-3958-4870-9a40-b18c78cf99aa req-6cc87ae4-9238-45ae-be48-216cd7ff030c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:24.885924 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-02056502-3958-4870-9a40-b18c78cf99aa req-6cc87ae4-9238-45ae-be48-216cd7ff030c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1582670 May 24 11:17:24.892670 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-f90be9e6-d4b6-4151-9e16-180c1afd7448 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:24.894087 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-f90be9e6-d4b6-4151-9e16-180c1afd7448 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 1096 time: 0.1252997 May 24 11:17:24.992946 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-7e092728-df18-427b-99ec-aba72e98d47d service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=159fe3b9-426c-472e-9870-2ead1792535a HTTP/1.1" status: 200 len: 217 time: 0.0893123 May 24 11:17:25.079299 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-02056502-3958-4870-9a40-b18c78cf99aa req-86e05688-9e83-4db5-9037-4bdbaa35ff57 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1743803 May 24 11:17:25.119831 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-b512bbbd-ad75-4a48-a87d-da8970c09ff5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:25.121748 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-b512bbbd-ad75-4a48-a87d-da8970c09ff5 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=1107b450-6c27-4f13-8b6e-abdbbbb1ea0f HTTP/1.1" status: 200 len: 854 time: 0.1125226 May 24 11:17:25.180420 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-1a762a58-0b05-4058-85d0-9eda4574fd2a service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0483925 May 24 11:17:25.380224 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-70b12605-eb67-4d59-8112-86da4e15c6d8 service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1855276 May 24 11:17:25.604955 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4dcdd1bb-1427-40b4-bb9d-92f9f9dce6a3 req-da86afb2-0ac0-48f5-847b-ac46faaf4743 service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2151687 May 24 11:17:25.624099 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d6fffa7c-7d16-4ccd-93fb-85bbf27cabd0 req-d55cd9d9-6a7e-4315-88d2-45c0ac5fb42e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:25.625627 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d6fffa7c-7d16-4ccd-93fb-85bbf27cabd0 req-d55cd9d9-6a7e-4315-88d2-45c0ac5fb42e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 943 time: 0.1594226 May 24 11:17:25.682820 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d6fffa7c-7d16-4ccd-93fb-85bbf27cabd0 req-45e78763-ef63-4707-80e6-c75a7ba74a3c tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0502949 May 24 11:17:25.749123 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:25.858416 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-206e6f41-c13c-43a4-b26a-f0fe47d1948f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:25.883727 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54246) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:26.005473 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a28c1c64-c5d1-4053-b6eb-4a070605d220 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:26.020316 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-a28c1c64-c5d1-4053-b6eb-4a070605d220 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:26.024150 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-a28c1c64-c5d1-4053-b6eb-4a070605d220 admin admin] Usage tracker for resource:security_group_rule and tenant:81e47a0e0f07448a95022f9a8131c5a9 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:26.027048 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-a28c1c64-c5d1-4053-b6eb-4a070605d220 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:26.038348 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-a28c1c64-c5d1-4053-b6eb-4a070605d220 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:26.133829 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a28c1c64-c5d1-4053-b6eb-4a070605d220 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:26.137696 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a28c1c64-c5d1-4053-b6eb-4a070605d220 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:26.145032 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0f90a29c-0790-4713-be80-dec540dd0df4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:26.183784 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a28c1c64-c5d1-4053-b6eb-4a070605d220 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=045a2302-c68e-41b1-b49d-42960c6638e1,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:26.186909 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-a28c1c64-c5d1-4053-b6eb-4a070605d220 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.042s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:26.189905 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-a28c1c64-c5d1-4053-b6eb-4a070605d220 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=81e47a0e0f07448a95022f9a8131c5a9&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3051353 May 24 11:17:26.192929 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54256) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:26.252268 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-972af683-cd5f-49fd-b583-a6aab2890364 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:26.284675 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f659e7ea-79e0-4399-8a4b-0efcf068c094 req-d3c60ecf-4a18-4bc7-b93d-7cbd9ac0a9e0 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:26.286717 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f659e7ea-79e0-4399-8a4b-0efcf068c094 req-d3c60ecf-4a18-4bc7-b93d-7cbd9ac0a9e0 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0888340 May 24 11:17:26.293476 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-972af683-cd5f-49fd-b583-a6aab2890364 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:26.340922 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f659e7ea-79e0-4399-8a4b-0efcf068c094 req-a527a1cf-4a67-4212-888a-e7fc1612f158 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0427549 May 24 11:17:26.356603 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-972af683-cd5f-49fd-b583-a6aab2890364 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:26.360445 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-824e4b4a-2983-48dd-9261-de7b280e41a9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:26.374256 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-972af683-cd5f-49fd-b583-a6aab2890364 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=045a2302-c68e-41b1-b49d-42960c6638e1,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:26.377147 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-972af683-cd5f-49fd-b583-a6aab2890364 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:26.379356 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-972af683-cd5f-49fd-b583-a6aab2890364 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:26.381044 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-972af683-cd5f-49fd-b583-a6aab2890364 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/045a2302-c68e-41b1-b49d-42960c6638e1 HTTP/1.1" status: 204 len: 168 time: 0.1867733 May 24 11:17:26.755377 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54276) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:26.775849 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-746953a5-0c51-47a5-9a86-dc11ae77493e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:26.792920 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-746953a5-0c51-47a5-9a86-dc11ae77493e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:26.797505 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-746953a5-0c51-47a5-9a86-dc11ae77493e admin admin] Usage tracker for resource:security_group_rule and tenant:8731631e6dfb45bd8379c3c4c4db87b8 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:26.804831 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-746953a5-0c51-47a5-9a86-dc11ae77493e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:26.817137 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-746953a5-0c51-47a5-9a86-dc11ae77493e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:26.938106 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aceb1697-b1e9-4d5d-933f-8a13e727ce3c req-76e0d234-84df-4bf7-a2ec-97954dcda0c1 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:26.946527 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-aceb1697-b1e9-4d5d-933f-8a13e727ce3c req-76e0d234-84df-4bf7-a2ec-97954dcda0c1 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 943 time: 0.1094813 May 24 11:17:26.956207 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:26.967414 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-746953a5-0c51-47a5-9a86-dc11ae77493e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:26.972790 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-746953a5-0c51-47a5-9a86-dc11ae77493e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:26.985759 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6467465b-0b2c-4a37-9e4c-496171fb1c5d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:27.010347 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-aceb1697-b1e9-4d5d-933f-8a13e727ce3c req-c556034d-a85f-44a1-95a8-d507ff040f27 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0620186 May 24 11:17:27.026326 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-746953a5-0c51-47a5-9a86-dc11ae77493e admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=8731631e6dfb45bd8379c3c4c4db87b8&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2695937 May 24 11:17:27.030413 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 54286) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:27.032115 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-746953a5-0c51-47a5-9a86-dc11ae77493e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a031d078-af38-45c9-8f71-f94e9b3f5846,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:27.037904 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-746953a5-0c51-47a5-9a86-dc11ae77493e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:27.106841 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dea4fa2d-911c-4095-914e-7d76f2ed44ce admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:27.140965 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dea4fa2d-911c-4095-914e-7d76f2ed44ce admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:27.204874 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dea4fa2d-911c-4095-914e-7d76f2ed44ce admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:27.209650 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:27.228688 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dea4fa2d-911c-4095-914e-7d76f2ed44ce admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a031d078-af38-45c9-8f71-f94e9b3f5846,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:27.231710 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-dea4fa2d-911c-4095-914e-7d76f2ed44ce admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:27.242539 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dea4fa2d-911c-4095-914e-7d76f2ed44ce admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:27.243717 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-dea4fa2d-911c-4095-914e-7d76f2ed44ce admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/a031d078-af38-45c9-8f71-f94e9b3f5846 HTTP/1.1" status: 204 len: 168 time: 0.2120221 May 24 11:17:27.558591 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb290a43-fa5a-47e8-b556-50a5568168a3 req-c50583b7-2874-4b0a-a489-dfac00f86919 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:27.572763 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cb290a43-fa5a-47e8-b556-50a5568168a3 req-c50583b7-2874-4b0a-a489-dfac00f86919 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0981004 May 24 11:17:27.629189 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cb290a43-fa5a-47e8-b556-50a5568168a3 req-079cfe1f-0724-48e9-8a81-8527f954b49e tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0469639 May 24 11:17:28.207195 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f197aa9e-16a7-4603-be76-94626e49da82 req-60887fe4-3f52-4848-8d2e-e1b5d3b37c80 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:28.208914 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f197aa9e-16a7-4603-be76-94626e49da82 req-60887fe4-3f52-4848-8d2e-e1b5d3b37c80 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 943 time: 0.0901296 May 24 11:17:28.258882 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f197aa9e-16a7-4603-be76-94626e49da82 req-4d971373-1a0d-4c2b-bce8-182410a215e3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0436049 May 24 11:17:28.846206 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70d58fd0-a41c-4638-80cc-20195cb3e2fc req-7951b3af-d09d-42cb-800d-720f8f2478f2 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:28.849314 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-70d58fd0-a41c-4638-80cc-20195cb3e2fc req-7951b3af-d09d-42cb-800d-720f8f2478f2 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0925055 May 24 11:17:28.879005 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54354) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:28.902124 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:28.921358 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-70d58fd0-a41c-4638-80cc-20195cb3e2fc req-ba55c746-6666-49eb-9e05-62c86c2fbf10 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0594234 May 24 11:17:29.028515 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8a269b9f-3d1d-49e7-9115-5b0fcafd70ad admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:29.047542 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-8a269b9f-3d1d-49e7-9115-5b0fcafd70ad admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:29.051401 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-8a269b9f-3d1d-49e7-9115-5b0fcafd70ad admin admin] Usage tracker for resource:security_group_rule and tenant:cb414df951e94f89bb198e0fe5d8db08 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:29.054250 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-8a269b9f-3d1d-49e7-9115-5b0fcafd70ad admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:29.059945 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-8a269b9f-3d1d-49e7-9115-5b0fcafd70ad admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:29.147812 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8a269b9f-3d1d-49e7-9115-5b0fcafd70ad admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:29.153516 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-8a269b9f-3d1d-49e7-9115-5b0fcafd70ad admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:29.161792 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-cc02d756-3752-4aa2-8cb5-4037f6c443b6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:29.203888 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8a269b9f-3d1d-49e7-9115-5b0fcafd70ad admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6ac1b34c-affc-4842-981b-cc344be83085,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:29.210418 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-8a269b9f-3d1d-49e7-9115-5b0fcafd70ad admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=cb414df951e94f89bb198e0fe5d8db08&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3274989 May 24 11:17:29.210418 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-8a269b9f-3d1d-49e7-9115-5b0fcafd70ad admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:29.210694 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54362) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:29.276150 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-4afe96ec-9249-4330-8588-0ba0bcab381d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:29.306676 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-4afe96ec-9249-4330-8588-0ba0bcab381d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:29.363692 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-4afe96ec-9249-4330-8588-0ba0bcab381d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:29.369347 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:29.385514 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4afe96ec-9249-4330-8588-0ba0bcab381d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6ac1b34c-affc-4842-981b-cc344be83085,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:29.389524 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-4afe96ec-9249-4330-8588-0ba0bcab381d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:29.397185 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-4afe96ec-9249-4330-8588-0ba0bcab381d admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:29.398295 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-4afe96ec-9249-4330-8588-0ba0bcab381d admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/6ac1b34c-affc-4842-981b-cc344be83085 HTTP/1.1" status: 204 len: 168 time: 0.1865408 May 24 11:17:29.479635 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4d31b98c-04be-44b9-b8d0-cf59b5c47dfc req-466e019d-1d7f-44f0-b6fc-944e6086529e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:29.482090 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4d31b98c-04be-44b9-b8d0-cf59b5c47dfc req-466e019d-1d7f-44f0-b6fc-944e6086529e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 943 time: 0.1077456 May 24 11:17:29.533407 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4d31b98c-04be-44b9-b8d0-cf59b5c47dfc req-7d440a0e-0caf-40a4-962f-811fd7244949 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0449057 May 24 11:17:29.988218 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 159fe3b9-426c-472e-9870-2ead1792535a details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:17:30.234822 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7de5e4be-8ff2-4ad6-8785-b3f5ae687580 req-bc0d4f53-2ada-40a9-95c4-ecf6ff5b089a tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:30.238313 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 159fe3b9-426c-472e-9870-2ead1792535a, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=159fe3b9-426c-472e-9870-2ead1792535a,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:17:30.243825 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7de5e4be-8ff2-4ad6-8785-b3f5ae687580 req-bc0d4f53-2ada-40a9-95c4-ecf6ff5b089a tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1743524 May 24 11:17:30.259698 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '159fe3b9-426c-472e-9870-2ead1792535a', 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add', 'port_id': '159fe3b9-426c-472e-9870-2ead1792535a', 'mac_address': 'fa:16:3e:fa:a5:1a', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 524, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '1107b450-6c27-4f13-8b6e-abdbbbb1ea0f', 'ip_address': '172.20.0.4'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:17:30.325269 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7de5e4be-8ff2-4ad6-8785-b3f5ae687580 req-e3a34ab5-0804-465f-9efb-00426ac302f3 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0804579 May 24 11:17:30.380946 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:30.399660 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 159fe3b9-426c-472e-9870-2ead1792535a {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:17:30.429732 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 159fe3b9-426c-472e-9870-2ead1792535a, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=159fe3b9-426c-472e-9870-2ead1792535a,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:17:30.436857 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:30.439902 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c9771ac0-1c52-4756-978f-810940b6daed None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:30.517536 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=159fe3b9-426c-472e-9870-2ead1792535a,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:30.525851 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:30.526205 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:30.526663 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:30.527000 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:30.527420 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:30.527623 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:30.528112 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:30.528431 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:30.528773 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:30.529108 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:30.529427 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:30.529755 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:30.534357 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:30.756350 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['159fe3b9-426c-472e-9870-2ead1792535a'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:17:30.766165 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a87d11e4-64ab-4883-bcd5-263d5a822879 req-4adc7613-2905-49f2-b8e2-d9c9289f1c8f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:30.772778 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a87d11e4-64ab-4883-bcd5-263d5a822879 req-4adc7613-2905-49f2-b8e2-d9c9289f1c8f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 944 time: 0.1052790 May 24 11:17:30.824557 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a87d11e4-64ab-4883-bcd5-263d5a822879 req-0f53dd47-7521-464e-ba6a-9812c4a060dd tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0506425 May 24 11:17:31.339149 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54438) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:31.407855 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_v2 [None req-2a6a762d-ba6f-4851-9dba-d6fefd29e514 admin admin] Deleting subnet d160e17b-87b6-4e4e-a58c-1a5ce3d62fd4 {{(pid=91820) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:17:31.411246 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2a6a762d-ba6f-4851-9dba-d6fefd29e514 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:31.476921 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2a6a762d-ba6f-4851-9dba-d6fefd29e514 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534861947'] for subnet, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:31.552092 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4d9111b2-8265-41c4-8ce6-b030089e3919 req-d20a429a-d5ca-4132-910a-1cb70e08cefb tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:31.562247 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4d9111b2-8265-41c4-8ce6-b030089e3919 req-d20a429a-d5ca-4132-910a-1cb70e08cefb tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1028554 May 24 11:17:31.562981 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2a6a762d-ba6f-4851-9dba-d6fefd29e514 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:31.631263 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4d9111b2-8265-41c4-8ce6-b030089e3919 req-b778212e-0da1-4f00-b6ca-a49ddb6d1ed8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0679708 May 24 11:17:31.985832 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 159fe3b9-426c-472e-9870-2ead1792535a up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:17:32.037071 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2a6a762d-ba6f-4851-9dba-d6fefd29e514 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534788488', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535456750'] for subnet, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.046812 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1ee2fcec-d7cc-4743-a98b-9dd4ac7b87ac admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:32.095026 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6963afb1-0536-4b05-ab2d-6727c2a320b7 req-e628d7d9-7c6d-4a45-8318-8cce5395dfbe tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:32.096140 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2a6a762d-ba6f-4851-9dba-d6fefd29e514 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.098000 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6963afb1-0536-4b05-ab2d-6727c2a320b7 req-e628d7d9-7c6d-4a45-8318-8cce5395dfbe tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 944 time: 0.1306765 May 24 11:17:32.099840 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 159fe3b9-426c-472e-9870-2ead1792535a completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:17:32.100883 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2a6a762d-ba6f-4851-9dba-d6fefd29e514 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=d160e17b-87b6-4e4e-a58c-1a5ce3d62fd4,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:32.105943 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-2a6a762d-ba6f-4851-9dba-d6fefd29e514 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:32.113418 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-2a6a762d-ba6f-4851-9dba-d6fefd29e514 admin admin] 10.222.0.18 "DELETE /v2.0/subnets/d160e17b-87b6-4e4e-a58c-1a5ce3d62fd4 HTTP/1.1" status: 204 len: 168 time: 0.7693231 May 24 11:17:32.113780 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54462) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:32.119535 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 159fe3b9-426c-472e-9870-2ead1792535a triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:17:32.119535 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.181957 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6963afb1-0536-4b05-ab2d-6727c2a320b7 req-423fb5d2-1fcd-468e-91df-5921fb47fc5a tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0767882 May 24 11:17:32.279272 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.299481 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.332176 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'd895da25-06b9-484c-8192-a6e4df06a856', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '159fe3b9-426c-472e-9870-2ead1792535a'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:17:32.343548 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 159fe3b9-426c-472e-9870-2ead1792535a, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=159fe3b9-426c-472e-9870-2ead1792535a,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:17:32.349143 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.351713 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:32.385590 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'd895da25-06b9-484c-8192-a6e4df06a856', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '159fe3b9-426c-472e-9870-2ead1792535a', 'code': 200} May 24 11:17:32.422958 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.433580 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=159fe3b9-426c-472e-9870-2ead1792535a,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:32.437391 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:32.446141 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['98ea8460-214e-4f24-a638-0313bfc40de5']}} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:32.466766 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'beb69048-046c-49d6-a095-eb202a63ca1d') {'for_net_delete': True} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:32.467022 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'beb69048-046c-49d6-a095-eb202a63ca1d') {} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:32.480864 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534861936'] for segment, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.507107 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.514926 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] Releasing vlan 526 on physical network physnet1 to pool {{(pid=91820) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:17:32.515557 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535262766'] for segment, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: Traceback (most recent call last): May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: timer() May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: cb(*args, **kw) May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: return func(*args, **kwargs) May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: self._notify() May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: self.callback(batched_events) May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: event.method(event) May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: return f(self, *a, **k) May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._get(url).json() May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: return self.request(url, 'GET', **kwargs) May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:17:32.533949 dv-cinder-784092-9-iscsi neutron-server[91820]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:17:32.637157 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.642317 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:32.655854 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.666806 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] Pushing event deleted for resources: {'Network': ['ID=98ea8460-214e-4f24-a638-0313bfc40de5,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:32.669482 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:32.672896 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] 10.222.0.18 "DELETE /v2.0/networks/98ea8460-214e-4f24-a638-0313bfc40de5 HTTP/1.1" status: 204 len: 168 time: 0.5557327 May 24 11:17:32.674322 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 54474) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:32.741153 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_v2 [None req-a7341bf9-d128-490a-bb94-61d655ce9e1b admin admin] Deleting subnet 51c7b9d7-d7c7-4132-b656-203c2f9e230a {{(pid=91819) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:17:32.744653 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a7341bf9-d128-490a-bb94-61d655ce9e1b admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.845569 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a7341bf9-d128-490a-bb94-61d655ce9e1b admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534853172'] for subnet, precommit_delete_associations {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.917608 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a7341bf9-d128-490a-bb94-61d655ce9e1b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:32.970395 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-078f6702-4dd8-4c5e-8967-153b9b85da11 req-707ed44e-5eb3-43b3-8a67-ab36e19cab93 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:32.971912 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-078f6702-4dd8-4c5e-8967-153b9b85da11 req-707ed44e-5eb3-43b3-8a67-ab36e19cab93 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.2012744 May 24 11:17:33.104544 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-078f6702-4dd8-4c5e-8967-153b9b85da11 req-0aba09b6-621a-47f6-85a3-98fa969afabd tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1079237 May 24 11:17:33.292212 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:17:33.317717 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a7341bf9-d128-490a-bb94-61d655ce9e1b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534944900', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535570805'] for subnet, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:33.323294 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:33.482177 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a7341bf9-d128-490a-bb94-61d655ce9e1b admin admin] Pushing event deleted for resources: {'Subnet': ['ID=51c7b9d7-d7c7-4132-b656-203c2f9e230a,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:33.484699 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a7341bf9-d128-490a-bb94-61d655ce9e1b admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:33.489860 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a7341bf9-d128-490a-bb94-61d655ce9e1b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.167s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:33.497701 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-a7341bf9-d128-490a-bb94-61d655ce9e1b admin admin] 10.222.0.18 "DELETE /v2.0/subnets/51c7b9d7-d7c7-4132-b656-203c2f9e230a HTTP/1.1" status: 204 len: 168 time: 0.8199701 May 24 11:17:33.498110 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54494) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:33.543532 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-47a149bd-7d0a-401a-9315-242aac7436ac req-81669fe3-9bab-4010-ad36-3a931a5f1ac0 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:33.545906 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-47a149bd-7d0a-401a-9315-242aac7436ac req-81669fe3-9bab-4010-ad36-3a931a5f1ac0 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 945 time: 0.1943128 May 24 11:17:33.601723 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-47a149bd-7d0a-401a-9315-242aac7436ac req-233dfb3a-2b3f-4f69-a5a0-45a82bcc4f2d tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0490980 May 24 11:17:33.613672 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:33.748582 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:33.766839 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['c24a20e8-37d4-4a0c-b252-5fc58f34aac1']}} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:33.780668 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '140422ff-fbc9-4c7d-874b-339405af2dc7') {'for_net_delete': True} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:33.780957 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '140422ff-fbc9-4c7d-874b-339405af2dc7') {} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:17:33.793736 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534861936'] for segment, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:33.810887 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:33.817204 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] Releasing vlan 531 on physical network physnet1 to pool {{(pid=91820) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:17:33.817586 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535262766'] for segment, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: Traceback (most recent call last): May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: timer() May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: cb(*args, **kw) May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: return func(*args, **kwargs) May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: self._notify() May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: self.callback(batched_events) May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: event.method(event) May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: return f(self, *a, **k) May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._get(url).json() May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: return self.request(url, 'GET', **kwargs) May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:17:33.831038 dv-cinder-784092-9-iscsi neutron-server[91820]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:17:33.916258 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:33.920175 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:33.941170 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:33.948553 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] Pushing event deleted for resources: {'Network': ['ID=c24a20e8-37d4-4a0c-b252-5fc58f34aac1,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:33.951079 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:33.954046 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] 10.222.0.18 "DELETE /v2.0/networks/c24a20e8-37d4-4a0c-b252-5fc58f34aac1 HTTP/1.1" status: 204 len: 168 time: 0.4535999 May 24 11:17:34.129923 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 54502) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:34.183426 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-2f4c00cc-d36c-406e-9e23-6bfa31349a2b admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=0dc56b1d592a4beea17c8068d8779718&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0521336 May 24 11:17:34.186406 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 54508) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:34.299942 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-32bfb5c4-43a8-4fca-89be-23b335841d3e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:34.335028 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-32bfb5c4-43a8-4fca-89be-23b335841d3e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:34.386837 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d76c803b-438f-4896-8e76-6377b5b1a47f req-1abf9dfb-40e3-403e-8057-05b39b209893 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:34.388332 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d76c803b-438f-4896-8e76-6377b5b1a47f req-1abf9dfb-40e3-403e-8057-05b39b209893 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1643627 May 24 11:17:34.434751 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-32bfb5c4-43a8-4fca-89be-23b335841d3e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:34.440946 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ed423585-6395-45c0-ae8a-3cf3a186c9be admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:34.452659 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d76c803b-438f-4896-8e76-6377b5b1a47f req-d3d72874-083f-45ba-b8de-afa6aa3e7a12 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0575821 May 24 11:17:34.460118 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-32bfb5c4-43a8-4fca-89be-23b335841d3e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=49092711-990a-4d12-82d5-1ac9ef151ab3,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:34.463267 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-32bfb5c4-43a8-4fca-89be-23b335841d3e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:34.465509 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-32bfb5c4-43a8-4fca-89be-23b335841d3e admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:34.466542 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-32bfb5c4-43a8-4fca-89be-23b335841d3e admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/49092711-990a-4d12-82d5-1ac9ef151ab3 HTTP/1.1" status: 204 len: 168 time: 0.2792375 May 24 11:17:34.808157 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4bac1b0f-da95-4d68-b5f2-58c913e1b33b req-1db44ef4-3009-4ac3-b10a-eb896dd9b400 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:34.809608 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4bac1b0f-da95-4d68-b5f2-58c913e1b33b req-1db44ef4-3009-4ac3-b10a-eb896dd9b400 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 945 time: 0.0995796 May 24 11:17:34.839127 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54524) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:34.862280 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4bac1b0f-da95-4d68-b5f2-58c913e1b33b req-464ba47b-95a5-48ac-8801-32ecd94623eb tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0470934 May 24 11:17:34.897914 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-cacf4b46-fd9f-4c3c-b309-437f20a77c4d admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=3e4e2c9d116d40dc825ed63a0bf2f2d1&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0576031 May 24 11:17:34.900382 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54528) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:34.962665 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1dc6f69b-2f84-4632-a944-6f16270e2d5d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:34.992163 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1dc6f69b-2f84-4632-a944-6f16270e2d5d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:35.050740 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1dc6f69b-2f84-4632-a944-6f16270e2d5d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:35.055257 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d2a3bad9-93d6-4ed8-a8fa-2f77cbc41fde admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:35.066388 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1dc6f69b-2f84-4632-a944-6f16270e2d5d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=18863090-0523-4364-bad8-a74b1bc86112,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:35.069045 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1dc6f69b-2f84-4632-a944-6f16270e2d5d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:35.078104 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1dc6f69b-2f84-4632-a944-6f16270e2d5d admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:35.079826 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-1dc6f69b-2f84-4632-a944-6f16270e2d5d admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/18863090-0523-4364-bad8-a74b1bc86112 HTTP/1.1" status: 204 len: 168 time: 0.1783578 May 24 11:17:35.651640 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49a6f4b0-bf11-47b6-a3f2-21c9c47e3b1b req-6e011f12-7c08-42ff-84c3-84ceab1536dc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:35.653357 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-49a6f4b0-bf11-47b6-a3f2-21c9c47e3b1b req-6e011f12-7c08-42ff-84c3-84ceab1536dc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0826828 May 24 11:17:35.704908 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-49a6f4b0-bf11-47b6-a3f2-21c9c47e3b1b req-3cf9b411-b687-4029-a923-2184a402cab1 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0456765 May 24 11:17:36.078968 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5da3b34a-9238-435f-8f1a-c09f3a642977 req-f2114f3f-f865-46e2-9204-b31899103349 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:36.086780 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5da3b34a-9238-435f-8f1a-c09f3a642977 req-f2114f3f-f865-46e2-9204-b31899103349 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 945 time: 0.1020677 May 24 11:17:36.109493 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a788641a-096b-4216-b3bd-dfa6bf3fd11b req-85f91e5a-2f35-4e85-ab33-d7015d747ee0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:36.112745 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a788641a-096b-4216-b3bd-dfa6bf3fd11b req-85f91e5a-2f35-4e85-ab33-d7015d747ee0 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 1098 time: 0.0948775 May 24 11:17:36.142363 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5da3b34a-9238-435f-8f1a-c09f3a642977 req-2cd6f874-6d22-497e-85d6-7e16bea34a8f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0542221 May 24 11:17:36.171352 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a788641a-096b-4216-b3bd-dfa6bf3fd11b req-4d16e1d9-1574-4fa6-a317-53ae26ac7b6f service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0492465 May 24 11:17:36.909483 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-334de094-1b6c-4e42-88e0-f50e82fa642f req-1470e9cb-5b19-4336-a9aa-0e80eac3a63e tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:36.911712 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-334de094-1b6c-4e42-88e0-f50e82fa642f req-1470e9cb-5b19-4336-a9aa-0e80eac3a63e tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0840814 May 24 11:17:36.962330 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-334de094-1b6c-4e42-88e0-f50e82fa642f req-1106dcc7-ed6d-4db4-a9b7-7751a8a6eade tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0444574 May 24 11:17:37.332993 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-00797b60-7e08-430b-9691-1531530d185b req-74dfd87d-3280-4260-ad74-96d720207369 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:37.334554 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-00797b60-7e08-430b-9691-1531530d185b req-74dfd87d-3280-4260-ad74-96d720207369 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 945 time: 0.0802293 May 24 11:17:37.381774 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-00797b60-7e08-430b-9691-1531530d185b req-74fe2186-c6de-4aae-b165-f1062b7484e3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0406027 May 24 11:17:38.002801 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 54622) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:38.142137 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-03ce19b6-053c-47fe-8f4c-2c24a5d7c8bc admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:38.157533 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-03ce19b6-053c-47fe-8f4c-2c24a5d7c8bc admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:38.161711 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-03ce19b6-053c-47fe-8f4c-2c24a5d7c8bc admin admin] Usage tracker for resource:security_group_rule and tenant:394001faec9b473f94beff9329c820e0 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:38.164825 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-03ce19b6-053c-47fe-8f4c-2c24a5d7c8bc admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:38.170939 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-03ce19b6-053c-47fe-8f4c-2c24a5d7c8bc admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:38.201890 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f202e410-93b9-49da-aa52-d2531b592c3f req-49a159b8-9f7d-4200-acf0-6b29e96250ef tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:38.203440 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f202e410-93b9-49da-aa52-d2531b592c3f req-49a159b8-9f7d-4200-acf0-6b29e96250ef tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0889902 May 24 11:17:38.268783 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f202e410-93b9-49da-aa52-d2531b592c3f req-b75593ba-22b8-40cb-aa3b-e4818b402bd2 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0571558 May 24 11:17:38.298110 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-03ce19b6-053c-47fe-8f4c-2c24a5d7c8bc admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:38.303460 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-03ce19b6-053c-47fe-8f4c-2c24a5d7c8bc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:38.316177 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-5b864378-33ac-4116-9533-f7fa2a0b873f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:38.412983 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-03ce19b6-053c-47fe-8f4c-2c24a5d7c8bc admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=394001faec9b473f94beff9329c820e0&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4138098 May 24 11:17:38.413922 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03ce19b6-053c-47fe-8f4c-2c24a5d7c8bc admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2eaed4c4-c316-4414-b6cb-312f0ea18e65,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:38.415881 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 54634) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:38.417031 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-03ce19b6-053c-47fe-8f4c-2c24a5d7c8bc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.101s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:38.486122 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a0aff458-cd50-4cb0-aadc-112ce5e6a564 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:38.548267 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a0aff458-cd50-4cb0-aadc-112ce5e6a564 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:38.628298 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-036884b9-3486-409e-8979-231a622ffb21 req-647f0acd-ef69-4c11-83e1-2e3f2f8826aa tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:38.630379 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-036884b9-3486-409e-8979-231a622ffb21 req-647f0acd-ef69-4c11-83e1-2e3f2f8826aa tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 945 time: 0.1226368 May 24 11:17:38.637342 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a0aff458-cd50-4cb0-aadc-112ce5e6a564 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:38.642660 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:38.672123 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a0aff458-cd50-4cb0-aadc-112ce5e6a564 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2eaed4c4-c316-4414-b6cb-312f0ea18e65,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:38.677941 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a0aff458-cd50-4cb0-aadc-112ce5e6a564 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:38.695860 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a0aff458-cd50-4cb0-aadc-112ce5e6a564 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:38.695860 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-a0aff458-cd50-4cb0-aadc-112ce5e6a564 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/2eaed4c4-c316-4414-b6cb-312f0ea18e65 HTTP/1.1" status: 204 len: 168 time: 0.2765710 May 24 11:17:38.706234 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-036884b9-3486-409e-8979-231a622ffb21 req-ddb376f8-e2d8-4cd1-85c6-f70883237d17 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0645111 May 24 11:17:39.058185 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 54654) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:39.078058 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-043b5f4f-2ea5-472b-8933-8ff3ef6ae76b admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:39.100698 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-043b5f4f-2ea5-472b-8933-8ff3ef6ae76b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:39.104995 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-043b5f4f-2ea5-472b-8933-8ff3ef6ae76b admin admin] Usage tracker for resource:security_group_rule and tenant:24ea88b99280401499519ac0be04be49 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:17:39.108725 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-043b5f4f-2ea5-472b-8933-8ff3ef6ae76b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:17:39.116424 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-043b5f4f-2ea5-472b-8933-8ff3ef6ae76b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:39.217721 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-043b5f4f-2ea5-472b-8933-8ff3ef6ae76b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:39.225125 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-043b5f4f-2ea5-472b-8933-8ff3ef6ae76b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:39.234211 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:39.278359 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-043b5f4f-2ea5-472b-8933-8ff3ef6ae76b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=045d5b08-bfbf-45f7-9cef-c9130e8e2698,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:39.282677 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-043b5f4f-2ea5-472b-8933-8ff3ef6ae76b admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=24ea88b99280401499519ac0be04be49&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2233362 May 24 11:17:39.283610 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-043b5f4f-2ea5-472b-8933-8ff3ef6ae76b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:39.285232 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 54662) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:17:39.369733 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cabd0f-e975-4d4b-803b-eae4689bf452 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:39.399235 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cabd0f-e975-4d4b-803b-eae4689bf452 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:39.466317 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cabd0f-e975-4d4b-803b-eae4689bf452 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:39.470712 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:39.482190 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03cabd0f-e975-4d4b-803b-eae4689bf452 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=045d5b08-bfbf-45f7-9cef-c9130e8e2698,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:39.489806 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-03cabd0f-e975-4d4b-803b-eae4689bf452 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:39.495423 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cabd0f-e975-4d4b-803b-eae4689bf452 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:39.499479 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-03cabd0f-e975-4d4b-803b-eae4689bf452 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/045d5b08-bfbf-45f7-9cef-c9130e8e2698 HTTP/1.1" status: 204 len: 168 time: 0.2126555 May 24 11:17:39.638149 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55d3065d-b342-4f3e-86e1-02ab81844a4c req-3787f825-7edc-457f-9816-9ad4386942d4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:39.639655 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-55d3065d-b342-4f3e-86e1-02ab81844a4c req-3787f825-7edc-457f-9816-9ad4386942d4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0865269 May 24 11:17:39.689071 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-55d3065d-b342-4f3e-86e1-02ab81844a4c req-b0ffd9ea-32af-4d9a-bd02-d65ab6ab7d07 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0435841 May 24 11:17:40.903223 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4d141d3e-0448-4dcf-81b8-59195d6edfd7 req-d4373423-21d6-4acd-b282-483d0c6fdf47 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:40.904734 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4d141d3e-0448-4dcf-81b8-59195d6edfd7 req-d4373423-21d6-4acd-b282-483d0c6fdf47 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0903046 May 24 11:17:40.955363 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4d141d3e-0448-4dcf-81b8-59195d6edfd7 req-1308fe27-1754-4ed3-9a2e-dbf967147f46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0450339 May 24 11:17:42.155519 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c46ca71e-31ee-4a10-b722-3e97bb58d617 req-50bba6c9-29e3-47c3-95b6-e13534acb4ec tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:42.157751 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c46ca71e-31ee-4a10-b722-3e97bb58d617 req-50bba6c9-29e3-47c3-95b6-e13534acb4ec tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0956860 May 24 11:17:42.206410 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c46ca71e-31ee-4a10-b722-3e97bb58d617 req-c3afbd4e-5012-47b7-81cc-e053a47f27d5 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0427406 May 24 11:17:43.422815 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85f2b514-09d1-44df-877c-8c53fe19269b req-9a8546d1-c783-4bb1-90ae-7cb98f353a8d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:43.424354 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-85f2b514-09d1-44df-877c-8c53fe19269b req-9a8546d1-c783-4bb1-90ae-7cb98f353a8d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0878360 May 24 11:17:43.493458 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-85f2b514-09d1-44df-877c-8c53fe19269b req-257193ce-e94f-43fd-a382-8dd5e1121de3 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0619345 May 24 11:17:44.691570 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6b05dcb6-8079-461a-afae-706e78a43cec req-5fe2e58d-2fa0-4c4d-a6d3-2b2e0ec997f1 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:44.693618 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6b05dcb6-8079-461a-afae-706e78a43cec req-5fe2e58d-2fa0-4c4d-a6d3-2b2e0ec997f1 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0871406 May 24 11:17:44.747119 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6b05dcb6-8079-461a-afae-706e78a43cec req-5c43b97f-e5a6-4480-a89f-4f1be04c2c74 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0475092 May 24 11:17:45.949703 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7fc68d36-8a61-4ffd-a74f-4c543d1d72ff req-342cef7d-fb5e-4d15-a9a8-28b1fc0e1e6f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:45.951207 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7fc68d36-8a61-4ffd-a74f-4c543d1d72ff req-342cef7d-fb5e-4d15-a9a8-28b1fc0e1e6f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0858231 May 24 11:17:46.004558 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7fc68d36-8a61-4ffd-a74f-4c543d1d72ff req-63c1a48d-4790-4910-a930-f6575bba554d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0473690 May 24 11:17:47.197659 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-77a277f4-30cc-4626-98ea-bf2265393986 req-15d44b0e-0f11-4411-ac02-6473c8650915 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:47.199462 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-77a277f4-30cc-4626-98ea-bf2265393986 req-15d44b0e-0f11-4411-ac02-6473c8650915 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0862517 May 24 11:17:47.252031 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-77a277f4-30cc-4626-98ea-bf2265393986 req-85e2cc96-aed8-493c-a53a-37abe9513ddc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0467801 May 24 11:17:48.529269 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08fbb4fc-07d4-4acf-aac0-a90d6511ee4e req-aa7392c3-e5a9-4114-8428-5233215dd9fc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:48.530637 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-08fbb4fc-07d4-4acf-aac0-a90d6511ee4e req-aa7392c3-e5a9-4114-8428-5233215dd9fc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1616659 May 24 11:17:48.577836 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-08fbb4fc-07d4-4acf-aac0-a90d6511ee4e req-63e1b376-7c5a-445f-ac25-1bb9730c3237 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0416074 May 24 11:17:49.799435 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2caea363-a8b1-40be-abb5-a78091161160 req-fd770b23-bd2a-4603-8ac6-c2b7d031c142 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:49.801012 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2caea363-a8b1-40be-abb5-a78091161160 req-fd770b23-bd2a-4603-8ac6-c2b7d031c142 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0933874 May 24 11:17:49.855103 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2caea363-a8b1-40be-abb5-a78091161160 req-67114a84-4a18-4cd3-85f0-58fdcc139b97 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0458016 May 24 11:17:50.390020 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5af3f05b-5836-4c8c-beb8-0b4bcbebbfb3 req-2c305d45-704c-4e33-b278-4e633a9bcda9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:50.391530 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5af3f05b-5836-4c8c-beb8-0b4bcbebbfb3 req-2c305d45-704c-4e33-b278-4e633a9bcda9 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 1099 time: 0.0737121 May 24 11:17:50.437143 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5af3f05b-5836-4c8c-beb8-0b4bcbebbfb3 req-f2d980a1-46ba-4162-bc5a-4cce9dcc3bb2 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0357900 May 24 11:17:50.976289 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-39ea6108-30a5-4800-8777-b43bfb2746be tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:50.976289 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-39ea6108-30a5-4800-8777-b43bfb2746be tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/networks?id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add HTTP/1.1" status: 200 len: 814 time: 0.1830966 May 24 11:17:50.996074 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-f53d9a20-437d-482e-b1ce-9b8094ba34ff tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/quotas/003eb49c73ae4e6a962cbfb425b3c1eb HTTP/1.1" status: 200 len: 375 time: 0.0115077 May 24 11:17:51.118869 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e2fec35-9a05-461b-9b18-06915521e0c7 req-32e8cef2-b476-4c9d-b744-b843a13ffc59 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:51.123297 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2e2fec35-9a05-461b-9b18-06915521e0c7 req-32e8cef2-b476-4c9d-b744-b843a13ffc59 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1141572 May 24 11:17:51.137210 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-0312a031-7f56-450a-8823-32a023c0faea tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&fields=id HTTP/1.1" status: 200 len: 302 time: 0.1333826 May 24 11:17:51.191311 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2e2fec35-9a05-461b-9b18-06915521e0c7 req-ca591405-c3e0-4036-985b-cd776a064986 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0670288 May 24 11:17:51.320018 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-15e3a899-58ce-46e8-b0e1-457e5bf07a6a service neutron] 10.222.0.18 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1807866 May 24 11:17:51.481957 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-32171dae-6332-45d7-92a8-a191f251d297 service neutron] 10.222.0.18 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1607590 May 24 11:17:51.692548 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7c09f4f9-ab0b-4c94-b3da-d1a1515736d3 req-65d0218d-f0c4-4c91-8f4e-c9ef6d6b503c tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0642135 May 24 11:17:51.975603 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:52.409917 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3306e114-fe9e-4d90-bd0a-760f7b4d556e req-9f098821-329f-4781-bdce-8ffe876c41df tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:52.411408 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3306e114-fe9e-4d90-bd0a-760f7b4d556e req-9f098821-329f-4781-bdce-8ffe876c41df tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.0826643 May 24 11:17:52.470012 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3306e114-fe9e-4d90-bd0a-760f7b4d556e req-c4bb6354-e53b-4e48-969b-9c0ecbcd4f96 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0527027 May 24 11:17:52.666167 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c9d9d3e-ea74-4bed-81bc-463c200a1a30 req-478886d4-496b-475c-9cf3-65c9a16a6ab0 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:52.667648 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4c9d9d3e-ea74-4bed-81bc-463c200a1a30 req-478886d4-496b-475c-9cf3-65c9a16a6ab0 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.0825129 May 24 11:17:52.713597 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4c9d9d3e-ea74-4bed-81bc-463c200a1a30 req-61344bcc-c908-47f7-9142-6ed99cd743d2 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0401294 May 24 11:17:52.859745 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-898a1013-6f88-4568-8711-bd0a1c637f6b req-a3628ba7-527f-42cc-9589-99998d020bcf tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0476739 May 24 11:17:53.405765 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-846a7cd3-ec4b-4356-8a40-5a72355c490f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:54.013104 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-bef9143d-5acc-4b02-bf15-f93975b860a6 req-199c2ea8-d7bc-49e8-8fa2-91d42f91bcd6 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0426028 May 24 11:17:55.179801 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e9bba173-f0e8-40ab-a13d-d21c47f94b8d req-1d527118-58ce-4876-95c2-7c064614a7a8 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0576241 May 24 11:17:55.745600 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:55.853960 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-deeacd45-eda1-4dc7-833d-1f6b29949de4 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:56.331368 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c0fff4a5-4af1-4d58-9f41-4319f6d81e61 req-42ddf787-4043-4156-b2d7-4ec7003001a1 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0460303 May 24 11:17:56.951487 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:57.486561 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f5607fff-99a7-40c1-abf7-9a12e692fc61 req-90cb02ad-d296-4da8-b5b9-f3351a2ab058 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0449018 May 24 11:17:58.642100 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-48f39718-6ab0-468f-8a89-499733b073c4 req-9b3d8636-320b-4877-9b25-7360530f2a65 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0442607 May 24 11:17:58.898571 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:58.926350 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-13f97e14-8f95-4e7b-9e80-0da8cabe5388 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:58.927668 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-13f97e14-8f95-4e7b-9e80-0da8cabe5388 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "GET /v2.0/networks?id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add HTTP/1.1" status: 200 len: 814 time: 0.1715097 May 24 11:17:58.942136 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Request body: {'port': {'device_id': '70697b65-6432-4d86-8536-934a7bf644ad', 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add', 'admin_state_up': True, 'tenant_id': '003eb49c73ae4e6a962cbfb425b3c1eb'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:17:59.087967 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:17:59.098166 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:17:59.108955 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Made reservation on behalf of 003eb49c73ae4e6a962cbfb425b3c1eb for: {'port': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:17:59.112991 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:59.333790 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Allocated IP 172.20.0.5 (bb3e0f6d-2046-4828-8f43-5f99ba5f3add/1107b450-6c27-4f13-8b6e-abdbbbb1ea0f/516f8f18-b5b5-4ded-bdfd-c420f5651d69) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:17:59.582845 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:59.617770 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:59.623643 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:17:59.688446 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Network bb3e0f6d-2046-4828-8f43-5f99ba5f3add is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:17:59.692195 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:17:59.696240 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:17:59.737437 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Pushing event updated for resources: {'Port': ['ID=516f8f18-b5b5-4ded-bdfd-c420f5651d69,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:17:59.751817 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:59.752168 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:59.752691 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:59.753216 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:59.753726 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:59.754751 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:59.755246 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:59.755737 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:59.756220 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:59.756699 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:59.757229 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:59.757722 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:17:59.767244 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.144s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:17:59.847835 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:59.848875 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:17:59.921339 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 0.9834714 May 24 11:17:59.936382 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0872abb9-1cbc-441b-a385-f7678a2a2125 req-1ea2d1ee-ac77-4669-9114-c2608429924f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:17:59.937189 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-0872abb9-1cbc-441b-a385-f7678a2a2125 req-1ea2d1ee-ac77-4669-9114-c2608429924f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Synchronizing usage tracker for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:17:59.954562 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-0872abb9-1cbc-441b-a385-f7678a2a2125 req-1ea2d1ee-ac77-4669-9114-c2608429924f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Unset dirty status for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:17:59.955568 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0872abb9-1cbc-441b-a385-f7678a2a2125 req-1ea2d1ee-ac77-4669-9114-c2608429924f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 931 time: 0.1840699 May 24 11:18:00.010573 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0872abb9-1cbc-441b-a385-f7678a2a2125 req-0873ffcb-f6ed-4070-8d35-2b2080d2f839 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0443890 May 24 11:18:00.522002 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['45911d6f-9595-40c5-8b29-75259c824bfe', '159fe3b9-426c-472e-9870-2ead1792535a'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:18:01.276265 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-038b90cd-5d50-4ad6-b4f0-c41754baceee req-8b729dc5-b9ca-4699-8650-7a9298b4fd01 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:01.277796 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-038b90cd-5d50-4ad6-b4f0-c41754baceee req-8b729dc5-b9ca-4699-8650-7a9298b4fd01 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 931 time: 0.1553121 May 24 11:18:01.338454 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-038b90cd-5d50-4ad6-b4f0-c41754baceee req-c08ef69b-845e-4ed1-9fe2-47085f7f983f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0548527 May 24 11:18:02.520677 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Request body: {'port': {'device_id': '70697b65-6432-4d86-8536-934a7bf644ad', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:02.631065 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1c3a4c0-a144-4ad7-a083-65168e85d155 req-a366179b-571b-4eff-8121-4333078a7a50 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:02.632942 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d1c3a4c0-a144-4ad7-a083-65168e85d155 req-a366179b-571b-4eff-8121-4333078a7a50 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 931 time: 0.1843410 May 24 11:18:02.692539 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:02.719804 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d1c3a4c0-a144-4ad7-a083-65168e85d155 req-83de1748-fb45-409e-bf07-42f7dd08266d tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0793884 May 24 11:18:03.088342 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 516f8f18-b5b5-4ded-bdfd-c420f5651d69 {{(pid=91820) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:18:03.096547 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:03.112762 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Transition to ACTIVE for port object 516f8f18-b5b5-4ded-bdfd-c420f5651d69 will not be triggered until provisioned by entity L2. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:03.117924 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:03.122282 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:03.324566 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Pushing event updated for resources: {'Port': ['ID=516f8f18-b5b5-4ded-bdfd-c420f5651d69,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:03.334650 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.212s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:03.435922 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Attempting to bind port 516f8f18-b5b5-4ded-bdfd-c420f5651d69 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:18:03.436460 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Attempting to bind port 516f8f18-b5b5-4ded-bdfd-c420f5651d69 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:18:03.436796 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Attempting to bind port 516f8f18-b5b5-4ded-bdfd-c420f5651d69 on network bb3e0f6d-2046-4828-8f43-5f99ba5f3add {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:03.443600 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Port 516f8f18-b5b5-4ded-bdfd-c420f5651d69 on network bb3e0f6d-2046-4828-8f43-5f99ba5f3add not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:18:03.443955 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Attempting to bind port 516f8f18-b5b5-4ded-bdfd-c420f5651d69 on network bb3e0f6d-2046-4828-8f43-5f99ba5f3add {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:03.450108 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 17, 52), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:18:03.450455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Checking segment: {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:03.450844 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Bound using segment: {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:18:03.451271 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Bound port: 516f8f18-b5b5-4ded-bdfd-c420f5651d69, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'}}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:18:03.451618 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:03.589408 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] For port 516f8f18-b5b5-4ded-bdfd-c420f5651d69, host hv-cinder-78409, cleared binding levels {{(pid=91820) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:18:03.594343 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] For port 516f8f18-b5b5-4ded-bdfd-c420f5651d69, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=516f8f18-b5b5-4ded-bdfd-c420f5651d69,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91820) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:18:03.645628 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:03.656672 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:03.718424 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d75706d9-5719-487e-9316-8745cf75561f req-04519e54-5f95-4497-a54f-8ca311adb034 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:03.720533 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d75706d9-5719-487e-9316-8745cf75561f req-04519e54-5f95-4497-a54f-8ca311adb034 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 1099 time: 0.1480961 May 24 11:18:03.765590 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:03.766202 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:03.767582 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] 10.222.0.93 "PUT /v2.0/ports/516f8f18-b5b5-4ded-bdfd-c420f5651d69 HTTP/1.1" status: 200 len: 1093 time: 1.2527895 May 24 11:18:03.768117 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '70697b65-6432-4d86-8536-934a7bf644ad', 'tag': '516f8f18-b5b5-4ded-bdfd-c420f5651d69'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:18:03.789517 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Pushing event updated for resources: {'Port': ['ID=516f8f18-b5b5-4ded-bdfd-c420f5651d69,revision_number=3']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:03.793138 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.137s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:03.811824 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-73fc0459-84e2-4f3d-aeaa-e9270267b833 req-dfbb2904-7d0a-4778-bf4c-6d67d0ddcdb3 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:03.813430 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-73fc0459-84e2-4f3d-aeaa-e9270267b833 req-dfbb2904-7d0a-4778-bf4c-6d67d0ddcdb3 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.2161224 May 24 11:18:03.882375 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-73fc0459-84e2-4f3d-aeaa-e9270267b833 req-654e583f-c704-4372-bbe3-1ebd66682dd5 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0623591 May 24 11:18:04.098521 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e6eb142-005c-4c1c-9958-da99beba524f req-2c514cf2-c325-4451-947f-193b5f5ef4fc tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:04.100548 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1e6eb142-005c-4c1c-9958-da99beba524f req-2c514cf2-c325-4451-947f-193b5f5ef4fc tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 943 time: 0.1070063 May 24 11:18:04.161056 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1e6eb142-005c-4c1c-9958-da99beba524f req-1b03c0bf-be8c-47f9-810e-bc84d0378171 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0477188 May 24 11:18:04.256159 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '70697b65-6432-4d86-8536-934a7bf644ad', 'tag': '516f8f18-b5b5-4ded-bdfd-c420f5651d69', 'status': 'completed', 'code': 200} May 24 11:18:04.471132 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 55102) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:04.628130 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:04.644098 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:04.648714 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Usage tracker for resource:security_group_rule and tenant:e020fd25ab1b4cc1afc0fa6fb1917950 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:04.651680 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:04.658654 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:04.753955 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:04.757817 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:04.765120 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-6081a293-3ddf-4e9c-b718-4db5af4f645f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:04.810495 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=e020fd25ab1b4cc1afc0fa6fb1917950&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3347266 May 24 11:18:04.810495 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=bcead26c-04b9-4e59-bef2-01074212bdb9,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:04.811260 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55114) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:04.815772 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:04.816134 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:04.816485 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:04.816874 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:04.817243 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:04.817585 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:04.817994 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:04.818337 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:04.818673 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:04.819004 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:04.819334 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:04.819665 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:04.823645 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:04.880461 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-81166790-def8-4626-ad3d-59f016bada5a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:04.909144 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-81166790-def8-4626-ad3d-59f016bada5a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:04.980528 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-81166790-def8-4626-ad3d-59f016bada5a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:04.988645 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:05.036605 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-81166790-def8-4626-ad3d-59f016bada5a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=bcead26c-04b9-4e59-bef2-01074212bdb9,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:05.040048 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-81166790-def8-4626-ad3d-59f016bada5a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:05.057099 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-81166790-def8-4626-ad3d-59f016bada5a admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:05.058225 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-81166790-def8-4626-ad3d-59f016bada5a admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/bcead26c-04b9-4e59-bef2-01074212bdb9 HTTP/1.1" status: 204 len: 168 time: 0.2460809 May 24 11:18:05.106699 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4dfb4417-bd4b-4b4d-8841-a33ce5eb498c req-47d6d424-3a1c-45a1-95d1-96c9516931b8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:05.108240 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4dfb4417-bd4b-4b4d-8841-a33ce5eb498c req-47d6d424-3a1c-45a1-95d1-96c9516931b8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1065147 May 24 11:18:05.167484 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4dfb4417-bd4b-4b4d-8841-a33ce5eb498c req-61de8be3-4bbf-44bb-a643-9288bc46eea2 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0518265 May 24 11:18:05.380508 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-84d6c7e6-d6ed-4b25-b5fa-fc0f49338aef req-37d97ce8-b840-4030-9e0d-cf63ed9537f3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:05.382048 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-84d6c7e6-d6ed-4b25-b5fa-fc0f49338aef req-37d97ce8-b840-4030-9e0d-cf63ed9537f3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 943 time: 0.0858088 May 24 11:18:05.429211 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-84d6c7e6-d6ed-4b25-b5fa-fc0f49338aef req-eefe6e24-99f5-4da7-9848-57597d41be3e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0410566 May 24 11:18:06.377251 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8f5eb320-b1b3-493e-add3-5ca3afac5cff req-25c9872b-11fe-4bca-942e-79397e6e49b7 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:06.378750 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8f5eb320-b1b3-493e-add3-5ca3afac5cff req-25c9872b-11fe-4bca-942e-79397e6e49b7 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.0951617 May 24 11:18:06.485619 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8f5eb320-b1b3-493e-add3-5ca3afac5cff req-e54bc925-f0a3-4aa3-b6c3-4c786d831635 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1004992 May 24 11:18:06.689734 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0d174096-4e64-4659-89ed-2977a6997665 req-fa6b53e0-08a0-4c17-8443-d1b1280dea17 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:06.691399 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0d174096-4e64-4659-89ed-2977a6997665 req-fa6b53e0-08a0-4c17-8443-d1b1280dea17 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 943 time: 0.0849268 May 24 11:18:06.793001 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0d174096-4e64-4659-89ed-2977a6997665 req-944c79ea-a5a3-4a2f-be9f-b0d9da5d0dcc tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0921478 May 24 11:18:06.883115 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1d8cf1d-fa35-4083-b7cc-967127401af5 req-746394bd-8df2-4fcc-9730-1e05bb4a79a6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:06.884577 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f1d8cf1d-fa35-4083-b7cc-967127401af5 req-746394bd-8df2-4fcc-9730-1e05bb4a79a6 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 1096 time: 0.0956137 May 24 11:18:06.928018 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d75706d9-5719-487e-9316-8745cf75561f req-a64e71a7-503c-4144-8c0a-5e46e2701c62 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:06.929643 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d75706d9-5719-487e-9316-8745cf75561f req-a64e71a7-503c-4144-8c0a-5e46e2701c62 service neutron] 10.222.0.93 "GET /v2.0/networks?id=4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 200 len: 918 time: 0.1880257 May 24 11:18:07.007369 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d75706d9-5719-487e-9316-8745cf75561f req-f8898be0-af26-4490-962f-772dc14010ed service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=02210ade-8c60-41b6-aa82-a104d4fc0447 HTTP/1.1" status: 200 len: 217 time: 0.0695903 May 24 11:18:07.043201 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1d8cf1d-fa35-4083-b7cc-967127401af5 req-007694c9-2a43-437c-b14f-40870f582562 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:07.044770 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f1d8cf1d-fa35-4083-b7cc-967127401af5 req-007694c9-2a43-437c-b14f-40870f582562 service neutron] 10.222.0.93 "GET /v2.0/networks?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&shared=False HTTP/1.1" status: 200 len: 915 time: 0.1511142 May 24 11:18:07.083733 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d75706d9-5719-487e-9316-8745cf75561f req-b1ebef8b-28c8-467e-8447-c3417da9d669 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:07.085251 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d75706d9-5719-487e-9316-8745cf75561f req-b1ebef8b-28c8-467e-8447-c3417da9d669 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=52267672-fd8c-45bd-89ea-d9d02a7b249a HTTP/1.1" status: 200 len: 857 time: 0.0633807 May 24 11:18:07.146091 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d75706d9-5719-487e-9316-8745cf75561f req-f6e0eb06-2d45-4890-9dc5-507073e05136 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=4fec6b68-64f1-4553-8ca5-f86c63ac9714&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0479610 May 24 11:18:07.195726 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1d8cf1d-fa35-4083-b7cc-967127401af5 req-41d27808-5a21-4c18-852b-79d2baab3737 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:07.197198 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f1d8cf1d-fa35-4083-b7cc-967127401af5 req-41d27808-5a21-4c18-852b-79d2baab3737 service neutron] 10.222.0.93 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1430266 May 24 11:18:07.311381 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d75706d9-5719-487e-9316-8745cf75561f req-a135d396-9a80-4a4e-b42f-cb5b666f6947 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1554399 May 24 11:18:07.336152 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-be9b7d71-3d32-49eb-8f3d-836cf7e448a6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:07.337688 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-be9b7d71-3d32-49eb-8f3d-836cf7e448a6 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 1096 time: 0.0870204 May 24 11:18:07.422018 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-0046e98f-dd4a-48a0-9c33-c1230bf13859 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=516f8f18-b5b5-4ded-bdfd-c420f5651d69 HTTP/1.1" status: 200 len: 217 time: 0.0696554 May 24 11:18:07.479592 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d75706d9-5719-487e-9316-8745cf75561f req-d86a11f0-d7a9-4eed-9cb2-02474b5400b4 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1585941 May 24 11:18:07.488126 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-2253ed0e-e54f-4576-9680-c2c97522548e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:07.489670 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-2253ed0e-e54f-4576-9680-c2c97522548e service neutron] 10.222.0.93 "GET /v2.0/subnets?id=1107b450-6c27-4f13-8b6e-abdbbbb1ea0f HTTP/1.1" status: 200 len: 854 time: 0.0607817 May 24 11:18:07.545910 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-e7800cd2-15cb-4fba-9b98-f11de7278abb service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0469644 May 24 11:18:07.706821 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7c3b2cfc-5551-4f31-9ef2-f08206388610 req-616d6496-bba0-4516-813a-1094b4cd6292 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:07.709557 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7c3b2cfc-5551-4f31-9ef2-f08206388610 req-616d6496-bba0-4516-813a-1094b4cd6292 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1070292 May 24 11:18:07.740985 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-8f846f27-7473-4dcc-87db-53209357fb4f service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1763523 May 24 11:18:07.772919 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7c3b2cfc-5551-4f31-9ef2-f08206388610 req-a30128a9-9888-42ad-80e1-8ac0f272f1cc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0486023 May 24 11:18:07.908580 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b0c8c9ca-c77c-42d9-9b12-054f6396560c req-9ab1a91b-0e1d-4f01-9dcd-31dee027fdc8 service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1571281 May 24 11:18:08.089824 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-62503fc6-f3cc-4ef1-b992-d9af9bd88cd3 req-e27a2d35-6c72-4612-b851-6d3c275e1103 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:08.096413 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-62503fc6-f3cc-4ef1-b992-d9af9bd88cd3 req-e27a2d35-6c72-4612-b851-6d3c275e1103 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 943 time: 0.1266091 May 24 11:18:08.265623 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-62503fc6-f3cc-4ef1-b992-d9af9bd88cd3 req-746ab641-d0ad-4f40-aad0-53bb263bfa92 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1682160 May 24 11:18:08.970932 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a707090b-f876-4949-9ff1-d2685136e540 req-481aa5dc-178f-41de-bca7-6de6facd80c3 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:08.972478 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a707090b-f876-4949-9ff1-d2685136e540 req-481aa5dc-178f-41de-bca7-6de6facd80c3 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.0823970 May 24 11:18:09.024805 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a707090b-f876-4949-9ff1-d2685136e540 req-1110c13f-fb0c-4f02-83d0-c0424a52c680 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0458846 May 24 11:18:09.476674 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-06b1e811-7ede-4442-a491-bfc56c2925c1 req-fb881022-bba4-46b0-9b20-b0880e53bad7 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:09.478127 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-06b1e811-7ede-4442-a491-bfc56c2925c1 req-fb881022-bba4-46b0-9b20-b0880e53bad7 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 943 time: 0.0892313 May 24 11:18:09.527056 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-06b1e811-7ede-4442-a491-bfc56c2925c1 req-9c0c63de-7f66-4a21-a471-8eae1db5187e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0425665 May 24 11:18:09.618617 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55220) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:09.762294 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-73bda772-80e8-4b18-a001-d3d225095b31 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:09.777781 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-73bda772-80e8-4b18-a001-d3d225095b31 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:09.782139 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-73bda772-80e8-4b18-a001-d3d225095b31 admin admin] Usage tracker for resource:security_group_rule and tenant:a5f8fb13fff74d54a7ea5ccd4845e952 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:09.785140 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-73bda772-80e8-4b18-a001-d3d225095b31 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:09.791239 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-73bda772-80e8-4b18-a001-d3d225095b31 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:09.895620 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-73bda772-80e8-4b18-a001-d3d225095b31 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:09.900989 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-73bda772-80e8-4b18-a001-d3d225095b31 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:09.908336 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1dc6f69b-2f84-4632-a944-6f16270e2d5d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:09.954339 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-73bda772-80e8-4b18-a001-d3d225095b31 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=a5f8fb13fff74d54a7ea5ccd4845e952&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3346298 May 24 11:18:09.955089 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-73bda772-80e8-4b18-a001-d3d225095b31 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0bb3bbbe-6913-459b-9fcc-ddb25cf41fae,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:09.957797 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 55228) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:09.958321 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-73bda772-80e8-4b18-a001-d3d225095b31 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:10.021387 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dd609af9-08d2-4503-9562-e83785e97e54 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:10.048234 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dd609af9-08d2-4503-9562-e83785e97e54 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:10.107376 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dd609af9-08d2-4503-9562-e83785e97e54 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:10.114510 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-7d2151b9-336c-49b2-8966-1ca2bd421f8e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:10.127151 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dd609af9-08d2-4503-9562-e83785e97e54 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0bb3bbbe-6913-459b-9fcc-ddb25cf41fae,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:10.131458 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-dd609af9-08d2-4503-9562-e83785e97e54 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:10.138192 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dd609af9-08d2-4503-9562-e83785e97e54 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:10.143221 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-dd609af9-08d2-4503-9562-e83785e97e54 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/0bb3bbbe-6913-459b-9fcc-ddb25cf41fae HTTP/1.1" status: 204 len: 168 time: 0.1803117 May 24 11:18:10.258311 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d564c98d-f91e-42b0-8b52-5bf9a1ebb66e req-8894696a-bd59-4d1e-9f79-93ce069bb9d8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:10.261620 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d564c98d-f91e-42b0-8b52-5bf9a1ebb66e req-8894696a-bd59-4d1e-9f79-93ce069bb9d8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1139789 May 24 11:18:10.295251 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:18:10.337904 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d564c98d-f91e-42b0-8b52-5bf9a1ebb66e req-862a3e9c-921a-4846-86f1-97fe95e5e788 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0697033 May 24 11:18:10.507948 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55240) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:10.525853 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1266b479-9648-4404-aadb-119bf686fd48 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:10.540499 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-1266b479-9648-4404-aadb-119bf686fd48 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:10.544907 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-1266b479-9648-4404-aadb-119bf686fd48 admin admin] Usage tracker for resource:security_group_rule and tenant:cd155f2542ad41f8b7e4beed5608a368 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:10.548213 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-1266b479-9648-4404-aadb-119bf686fd48 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:10.560512 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-1266b479-9648-4404-aadb-119bf686fd48 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:10.712199 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1266b479-9648-4404-aadb-119bf686fd48 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:10.725664 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1266b479-9648-4404-aadb-119bf686fd48 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:10.734415 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-8a269b9f-3d1d-49e7-9115-5b0fcafd70ad admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:10.778498 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-1266b479-9648-4404-aadb-119bf686fd48 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=cd155f2542ad41f8b7e4beed5608a368&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2694497 May 24 11:18:10.781697 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 55246) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:10.789004 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1266b479-9648-4404-aadb-119bf686fd48 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ed7b159e-7284-45e1-82dd-6473f91678e3,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:10.793657 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1266b479-9648-4404-aadb-119bf686fd48 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:10.810989 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ae2131e-7c57-4664-9973-f01ca125c46c req-f67e3ebb-751b-4fa2-a7cb-30024890f056 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:10.811911 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0ae2131e-7c57-4664-9973-f01ca125c46c req-f67e3ebb-751b-4fa2-a7cb-30024890f056 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 943 time: 0.1623399 May 24 11:18:10.855506 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-597dc802-9edc-4b32-ad82-fb2134f5e068 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:10.873299 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0ae2131e-7c57-4664-9973-f01ca125c46c req-62841b5c-f91d-466d-bc58-83696a96e90f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0547566 May 24 11:18:10.892003 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-597dc802-9edc-4b32-ad82-fb2134f5e068 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:10.945463 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-597dc802-9edc-4b32-ad82-fb2134f5e068 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:10.949531 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a0aff458-cd50-4cb0-aadc-112ce5e6a564 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:10.962637 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-597dc802-9edc-4b32-ad82-fb2134f5e068 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ed7b159e-7284-45e1-82dd-6473f91678e3,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:10.965586 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-597dc802-9edc-4b32-ad82-fb2134f5e068 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:10.970709 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-597dc802-9edc-4b32-ad82-fb2134f5e068 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:10.970709 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-597dc802-9edc-4b32-ad82-fb2134f5e068 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/ed7b159e-7284-45e1-82dd-6473f91678e3 HTTP/1.1" status: 204 len: 168 time: 0.1857941 May 24 11:18:11.617414 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-56740421-2244-431b-a734-af703f42a55e req-ad3213f0-4d50-4e28-9145-dc1dcf7a87b4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:11.618992 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-56740421-2244-431b-a734-af703f42a55e req-ad3213f0-4d50-4e28-9145-dc1dcf7a87b4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1650610 May 24 11:18:11.667350 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-56740421-2244-431b-a734-af703f42a55e req-e5c5da39-df88-49c6-aa3a-852dbbd0ec33 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0422595 May 24 11:18:12.074851 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b24c6210-2e55-44dd-862d-735f70c54394 req-7e4f7dbd-7f71-4f0e-91ca-4fe522843fa0 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:12.076479 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b24c6210-2e55-44dd-862d-735f70c54394 req-7e4f7dbd-7f71-4f0e-91ca-4fe522843fa0 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 943 time: 0.0897617 May 24 11:18:12.137706 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b24c6210-2e55-44dd-862d-735f70c54394 req-304fe804-f0c2-4cb4-8182-bf83f52690a7 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0554078 May 24 11:18:12.890783 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0ea5d9d-e351-43a7-ba92-69984103ed33 req-73c794c1-a9b7-4b16-a9a6-1b9b8e2f1985 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:12.892591 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e0ea5d9d-e351-43a7-ba92-69984103ed33 req-73c794c1-a9b7-4b16-a9a6-1b9b8e2f1985 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.0997415 May 24 11:18:13.002470 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e0ea5d9d-e351-43a7-ba92-69984103ed33 req-532563c4-3d21-47ae-804c-8cb9fc7a3969 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1031377 May 24 11:18:13.376699 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d51a50f0-c3aa-48b7-bc44-2bb8e23b49f2 req-e6caaeaa-daa1-4d9c-afe0-cc34ee4df46c tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:13.378081 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d51a50f0-c3aa-48b7-bc44-2bb8e23b49f2 req-e6caaeaa-daa1-4d9c-afe0-cc34ee4df46c tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 943 time: 0.1031783 May 24 11:18:13.442858 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d51a50f0-c3aa-48b7-bc44-2bb8e23b49f2 req-71ad3f81-54af-471e-a05d-48dcc61ce3a8 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0576789 May 24 11:18:14.225107 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-24f985a7-60d9-42ed-aa79-5e869b71c989 req-bfe2e3fa-fb1d-4208-9dbb-a9127d6bac79 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:14.226610 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-24f985a7-60d9-42ed-aa79-5e869b71c989 req-bfe2e3fa-fb1d-4208-9dbb-a9127d6bac79 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.0937841 May 24 11:18:14.277118 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-24f985a7-60d9-42ed-aa79-5e869b71c989 req-98bb6689-b733-430c-ad8e-65c77a1df48d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0447040 May 24 11:18:14.668559 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d85d5a3a-e2ce-4561-b6d5-bc893368131b req-3a612bfd-f13d-4c0d-8b03-b7573053c04e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:14.669691 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d85d5a3a-e2ce-4561-b6d5-bc893368131b req-3a612bfd-f13d-4c0d-8b03-b7573053c04e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 943 time: 0.0940747 May 24 11:18:14.722241 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d85d5a3a-e2ce-4561-b6d5-bc893368131b req-a4f44235-8505-4d86-9ec0-4578096d401c tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0466294 May 24 11:18:14.749670 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 516f8f18-b5b5-4ded-bdfd-c420f5651d69 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:18:14.907301 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 516f8f18-b5b5-4ded-bdfd-c420f5651d69, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=516f8f18-b5b5-4ded-bdfd-c420f5651d69,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:14.922158 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '516f8f18-b5b5-4ded-bdfd-c420f5651d69', 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add', 'port_id': '516f8f18-b5b5-4ded-bdfd-c420f5651d69', 'mac_address': 'fa:16:3e:93:e6:df', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 524, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '1107b450-6c27-4f13-8b6e-abdbbbb1ea0f', 'ip_address': '172.20.0.5'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:18:14.986258 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:15.011307 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 516f8f18-b5b5-4ded-bdfd-c420f5651d69 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:18:15.030715 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 516f8f18-b5b5-4ded-bdfd-c420f5651d69, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=516f8f18-b5b5-4ded-bdfd-c420f5651d69,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:15.034596 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:15.037828 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:15.118923 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=516f8f18-b5b5-4ded-bdfd-c420f5651d69,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:15.122446 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:15.358822 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['516f8f18-b5b5-4ded-bdfd-c420f5651d69'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:18:15.475384 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f391e228-ed99-4997-af5e-e37925a314dd req-0fd50ef8-cc9f-4158-bfde-b3a34bb383df tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:15.477657 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f391e228-ed99-4997-af5e-e37925a314dd req-0fd50ef8-cc9f-4158-bfde-b3a34bb383df tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.0868094 May 24 11:18:15.537866 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f391e228-ed99-4997-af5e-e37925a314dd req-27b8f6d6-7835-461a-90c6-987d9823db5d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0507159 May 24 11:18:15.959778 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-256b1f10-a8bf-479c-8e4a-67a57284e6e7 req-d9e18486-cf38-4cb7-9300-b94d90d123d8 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:15.962389 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-256b1f10-a8bf-479c-8e4a-67a57284e6e7 req-d9e18486-cf38-4cb7-9300-b94d90d123d8 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 944 time: 0.1011791 May 24 11:18:16.009723 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-256b1f10-a8bf-479c-8e4a-67a57284e6e7 req-21db933c-1525-4422-83f8-88d2dc4a22aa tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0424950 May 24 11:18:16.766529 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 516f8f18-b5b5-4ded-bdfd-c420f5651d69 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:16.784560 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d70db58d-dbdd-4153-8897-08364d7849e1 req-4a0587cb-adc2-489f-89f4-bc84920b0125 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:16.792672 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d70db58d-dbdd-4153-8897-08364d7849e1 req-4a0587cb-adc2-489f-89f4-bc84920b0125 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1212792 May 24 11:18:16.852164 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d70db58d-dbdd-4153-8897-08364d7849e1 req-9f5a6837-39e0-413a-a58a-5ecf9cc44785 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0581684 May 24 11:18:16.870733 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 516f8f18-b5b5-4ded-bdfd-c420f5651d69 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:16.877225 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 516f8f18-b5b5-4ded-bdfd-c420f5651d69 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:16.877788 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:17.118238 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:17.207063 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '70697b65-6432-4d86-8536-934a7bf644ad', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '516f8f18-b5b5-4ded-bdfd-c420f5651d69'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:18:17.226964 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 516f8f18-b5b5-4ded-bdfd-c420f5651d69, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=516f8f18-b5b5-4ded-bdfd-c420f5651d69,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:17.234435 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:17.240825 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:17.274265 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55430) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:17.377587 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '70697b65-6432-4d86-8536-934a7bf644ad', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '516f8f18-b5b5-4ded-bdfd-c420f5651d69', 'code': 200} May 24 11:18:17.460109 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=516f8f18-b5b5-4ded-bdfd-c420f5651d69,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:17.467902 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-04adb659-f816-4bed-93dd-33952c040896 req-0f622f76-5ceb-428a-a82f-decfc061e1c5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:17.469044 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.229s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:17.477977 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-04adb659-f816-4bed-93dd-33952c040896 req-0f622f76-5ceb-428a-a82f-decfc061e1c5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 945 time: 0.2342992 May 24 11:18:17.525596 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-21a16983-c74d-42f4-9d40-2423009a9a5b admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:17.551348 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-04adb659-f816-4bed-93dd-33952c040896 req-c12ae4fa-7c23-4274-a948-316606637bc8 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0688567 May 24 11:18:17.565030 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-21a16983-c74d-42f4-9d40-2423009a9a5b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:17.570495 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-21a16983-c74d-42f4-9d40-2423009a9a5b admin admin] Usage tracker for resource:security_group_rule and tenant:d5a9db8c1e3a407bbf478b23036dc501 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:17.574616 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-21a16983-c74d-42f4-9d40-2423009a9a5b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:17.613121 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-21a16983-c74d-42f4-9d40-2423009a9a5b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:17.703606 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49843143-131e-4a8a-a0cd-f572b1763b34 req-11293135-7c7a-4522-b7d2-53d8ebf0d1f8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:17.705472 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-49843143-131e-4a8a-a0cd-f572b1763b34 req-11293135-7c7a-4522-b7d2-53d8ebf0d1f8 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 1098 time: 0.1276073 May 24 11:18:17.760912 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55452) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:17.806214 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-21a16983-c74d-42f4-9d40-2423009a9a5b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:17.808530 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-49843143-131e-4a8a-a0cd-f572b1763b34 req-5e8565e1-41bd-4851-bea6-8d3fb859a1db service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0861027 May 24 11:18:17.812703 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-21a16983-c74d-42f4-9d40-2423009a9a5b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:17.820581 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b12eaabd-eee4-44c9-847e-9bb717392dc9 req-40bb2c96-4d63-4f85-a423-45e74d6cdd3e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:17.822301 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b12eaabd-eee4-44c9-847e-9bb717392dc9 req-40bb2c96-4d63-4f85-a423-45e74d6cdd3e service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 1099 time: 0.0989864 May 24 11:18:17.824987 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-04a73591-899b-47ad-ad9e-a078a5ac2f8b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:17.845117 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 55458) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:17.866908 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-21a16983-c74d-42f4-9d40-2423009a9a5b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7ff83a21-db48-4e7d-b223-dd9a28b450fe,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:17.875329 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-21a16983-c74d-42f4-9d40-2423009a9a5b admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=d5a9db8c1e3a407bbf478b23036dc501&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5965726 May 24 11:18:17.875329 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-21a16983-c74d-42f4-9d40-2423009a9a5b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:17.875607 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55466) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:17.968412 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ee0d752c-3c2a-4f39-807e-c09c3c6e85d7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.001318 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ee0d752c-3c2a-4f39-807e-c09c3c6e85d7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.067136 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ee0d752c-3c2a-4f39-807e-c09c3c6e85d7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.074040 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-21a16983-c74d-42f4-9d40-2423009a9a5b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:18.103586 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ee0d752c-3c2a-4f39-807e-c09c3c6e85d7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7ff83a21-db48-4e7d-b223-dd9a28b450fe,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:18.106641 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ee0d752c-3c2a-4f39-807e-c09c3c6e85d7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:18.112521 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e5e84ec5-3ef8-4b1e-8292-cbd4450907a8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.118406 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0a177670-65d5-403d-8a6b-cd05e0c2c716 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.143712 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-e5e84ec5-3ef8-4b1e-8292-cbd4450907a8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:18.149136 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-e5e84ec5-3ef8-4b1e-8292-cbd4450907a8 admin admin] Usage tracker for resource:security_group_rule and tenant:66e92a04986144f8b9e66a6009516625 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:18.152280 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ee0d752c-3c2a-4f39-807e-c09c3c6e85d7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.153450 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-ee0d752c-3c2a-4f39-807e-c09c3c6e85d7 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/7ff83a21-db48-4e7d-b223-dd9a28b450fe HTTP/1.1" status: 204 len: 168 time: 0.2771139 May 24 11:18:18.154537 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-e5e84ec5-3ef8-4b1e-8292-cbd4450907a8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:18.156962 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-0a177670-65d5-403d-8a6b-cd05e0c2c716 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:18.165321 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b12eaabd-eee4-44c9-847e-9bb717392dc9 req-b6952305-4672-4878-b1fb-7b45d167458e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:18.166889 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-e5e84ec5-3ef8-4b1e-8292-cbd4450907a8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:18.167056 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b12eaabd-eee4-44c9-847e-9bb717392dc9 req-b6952305-4672-4878-b1fb-7b45d167458e service neutron] 10.222.0.93 "GET /v2.0/networks?id=4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 200 len: 918 time: 0.2518702 May 24 11:18:18.169044 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-0a177670-65d5-403d-8a6b-cd05e0c2c716 admin admin] Usage tracker for resource:security_group_rule and tenant:78d72db4c5a64253bb3bc476e86d6d0d is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:18.174599 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-0a177670-65d5-403d-8a6b-cd05e0c2c716 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:18.244428 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-0a177670-65d5-403d-8a6b-cd05e0c2c716 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:18.256856 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b12eaabd-eee4-44c9-847e-9bb717392dc9 req-93de2979-e625-4bdf-99ee-c51fc31d0666 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=02210ade-8c60-41b6-aa82-a104d4fc0447 HTTP/1.1" status: 200 len: 217 time: 0.0784051 May 24 11:18:18.313069 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e5e84ec5-3ef8-4b1e-8292-cbd4450907a8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.318319 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e5e84ec5-3ef8-4b1e-8292-cbd4450907a8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.330464 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6592716c-238b-43a4-b44d-f06cbc8c459d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:18.345945 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dcda94ff-26ab-4fbb-b3b4-dbdbe3272422 req-e6d5c772-5982-4f56-9887-8a26736b546c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:18.349966 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dcda94ff-26ab-4fbb-b3b4-dbdbe3272422 req-e6d5c772-5982-4f56-9887-8a26736b546c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.3248599 May 24 11:18:18.378936 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e5e84ec5-3ef8-4b1e-8292-cbd4450907a8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a58d4476-4f56-4db4-a425-e8efed8ac2d9,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:18.383339 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-e5e84ec5-3ef8-4b1e-8292-cbd4450907a8 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=66e92a04986144f8b9e66a6009516625&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6209500 May 24 11:18:18.384385 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e5e84ec5-3ef8-4b1e-8292-cbd4450907a8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:18.385989 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55474) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:18.415005 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b12eaabd-eee4-44c9-847e-9bb717392dc9 req-53747c97-8bc6-4524-9fcb-68fe9c85cbc9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:18.416445 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b12eaabd-eee4-44c9-847e-9bb717392dc9 req-53747c97-8bc6-4524-9fcb-68fe9c85cbc9 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=52267672-fd8c-45bd-89ea-d9d02a7b249a HTTP/1.1" status: 200 len: 857 time: 0.1484258 May 24 11:18:18.455590 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5d9cf2b9-5770-44b9-a624-c0c67ca90e23 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.487365 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b12eaabd-eee4-44c9-847e-9bb717392dc9 req-c9e1e79f-e9b7-4d90-a25e-3fd4f99b0db4 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=4fec6b68-64f1-4553-8ca5-f86c63ac9714&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0552087 May 24 11:18:18.490542 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5d9cf2b9-5770-44b9-a624-c0c67ca90e23 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.495094 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dcda94ff-26ab-4fbb-b3b4-dbdbe3272422 req-700190cd-ded4-49cf-a211-bb705f0cf57a tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1400411 May 24 11:18:18.581231 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5d9cf2b9-5770-44b9-a624-c0c67ca90e23 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.587364 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-972af683-cd5f-49fd-b583-a6aab2890364 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:18.599795 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5d9cf2b9-5770-44b9-a624-c0c67ca90e23 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a58d4476-4f56-4db4-a425-e8efed8ac2d9,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:18.604834 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-5d9cf2b9-5770-44b9-a624-c0c67ca90e23 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:18.615771 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5d9cf2b9-5770-44b9-a624-c0c67ca90e23 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.617799 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-5d9cf2b9-5770-44b9-a624-c0c67ca90e23 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/a58d4476-4f56-4db4-a425-e8efed8ac2d9 HTTP/1.1" status: 204 len: 168 time: 0.2305923 May 24 11:18:18.641602 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0a177670-65d5-403d-8a6b-cd05e0c2c716 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.701344 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b12eaabd-eee4-44c9-847e-9bb717392dc9 req-ac38cb63-39c2-42e1-8ff9-4b64e06ea66c service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1988597 May 24 11:18:18.701344 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0a177670-65d5-403d-8a6b-cd05e0c2c716 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:18.754951 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5f71b330-1378-4f7e-9eff-b698164d183f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:18.785872 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-0a177670-65d5-403d-8a6b-cd05e0c2c716 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=78d72db4c5a64253bb3bc476e86d6d0d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.9394674 May 24 11:18:18.789919 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55492) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:18.808465 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0a177670-65d5-403d-8a6b-cd05e0c2c716 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d9d365fd-7ae9-4f59-9da2-bba8d45ce946,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:18.831736 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-0a177670-65d5-403d-8a6b-cd05e0c2c716 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:18.999388 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b12eaabd-eee4-44c9-847e-9bb717392dc9 req-3626e8f7-d600-4ec5-909d-1f36bc398107 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2468295 May 24 11:18:19.033415 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0e503538-937b-483c-bf34-ba38d5f97b0c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:19.050301 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eb838c06-7650-4236-a5e0-79e84cb94353 req-3e72aca1-dc1b-479a-8357-493677d54067 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:19.052466 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-eb838c06-7650-4236-a5e0-79e84cb94353 req-3e72aca1-dc1b-479a-8357-493677d54067 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 945 time: 0.3426790 May 24 11:18:19.091663 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0e503538-937b-483c-bf34-ba38d5f97b0c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:19.149327 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-eb838c06-7650-4236-a5e0-79e84cb94353 req-f1007700-91b1-4408-aceb-73342f7e5eb3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0845122 May 24 11:18:19.177371 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55504) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:19.197565 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-af563873-d3e2-46a6-a8b8-da4ff5dbbbaa admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:19.209356 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0e503538-937b-483c-bf34-ba38d5f97b0c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:19.215811 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-47c4ddc8-9771-439b-84eb-572930717615 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:19.229687 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0e503538-937b-483c-bf34-ba38d5f97b0c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d9d365fd-7ae9-4f59-9da2-bba8d45ce946,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:19.231788 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-af563873-d3e2-46a6-a8b8-da4ff5dbbbaa admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:19.239013 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0e503538-937b-483c-bf34-ba38d5f97b0c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:19.240030 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-af563873-d3e2-46a6-a8b8-da4ff5dbbbaa admin admin] Usage tracker for resource:security_group_rule and tenant:7f046381ed8c480c84795812e47c3c46 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:19.247334 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-af563873-d3e2-46a6-a8b8-da4ff5dbbbaa admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:19.256176 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0e503538-937b-483c-bf34-ba38d5f97b0c admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:19.260722 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-0e503538-937b-483c-bf34-ba38d5f97b0c admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/d9d365fd-7ae9-4f59-9da2-bba8d45ce946 HTTP/1.1" status: 204 len: 168 time: 0.4661036 May 24 11:18:19.260722 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-af563873-d3e2-46a6-a8b8-da4ff5dbbbaa admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:19.390482 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-af563873-d3e2-46a6-a8b8-da4ff5dbbbaa admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:19.396297 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-af563873-d3e2-46a6-a8b8-da4ff5dbbbaa admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:19.407717 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-746953a5-0c51-47a5-9a86-dc11ae77493e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:19.456720 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-af563873-d3e2-46a6-a8b8-da4ff5dbbbaa admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=7f046381ed8c480c84795812e47c3c46&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2782025 May 24 11:18:19.458355 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af563873-d3e2-46a6-a8b8-da4ff5dbbbaa admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6d782bad-598e-43b3-b1dc-9310ecdeb7d2,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:19.459301 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 55516) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:19.461777 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-af563873-d3e2-46a6-a8b8-da4ff5dbbbaa admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:19.626097 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-72a553be-ff04-49a1-b3d8-e15f10aa7ac1 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:19.709052 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-72a553be-ff04-49a1-b3d8-e15f10aa7ac1 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:19.784963 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6cfbba21-9563-4871-970d-54643c1366d3 req-52cc6548-de5a-42d3-a5fa-7b9291e7f00e tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:19.787217 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6cfbba21-9563-4871-970d-54643c1366d3 req-52cc6548-de5a-42d3-a5fa-7b9291e7f00e tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1361272 May 24 11:18:19.815590 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-72a553be-ff04-49a1-b3d8-e15f10aa7ac1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:19.822632 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-dea4fa2d-911c-4095-914e-7d76f2ed44ce admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:19.850641 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-72a553be-ff04-49a1-b3d8-e15f10aa7ac1 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6d782bad-598e-43b3-b1dc-9310ecdeb7d2,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:19.861355 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-72a553be-ff04-49a1-b3d8-e15f10aa7ac1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:19.878032 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6cfbba21-9563-4871-970d-54643c1366d3 req-9288d0a2-808f-481f-8e4b-4827f2b218ca tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0840240 May 24 11:18:19.881436 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-72a553be-ff04-49a1-b3d8-e15f10aa7ac1 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:19.883103 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-72a553be-ff04-49a1-b3d8-e15f10aa7ac1 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/6d782bad-598e-43b3-b1dc-9310ecdeb7d2 HTTP/1.1" status: 204 len: 168 time: 0.4205999 May 24 11:18:20.425353 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bdeb1918-fa7d-416a-a23f-000dd852fb31 req-6824b884-4d2a-4ebb-93d7-2f392b7167f3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:20.427377 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bdeb1918-fa7d-416a-a23f-000dd852fb31 req-6824b884-4d2a-4ebb-93d7-2f392b7167f3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 945 time: 0.1332572 May 24 11:18:20.489635 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bdeb1918-fa7d-416a-a23f-000dd852fb31 req-b6080319-19a1-4341-9359-6f75c58b13a9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0520954 May 24 11:18:21.239994 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c8cfe3fa-455e-42af-90f6-6fe024d507f2 req-f4abee99-698b-490d-bab4-afe89ee5f6a8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:21.246008 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c8cfe3fa-455e-42af-90f6-6fe024d507f2 req-f4abee99-698b-490d-bab4-afe89ee5f6a8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1845984 May 24 11:18:21.323512 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c8cfe3fa-455e-42af-90f6-6fe024d507f2 req-306c82ae-42ff-48e5-98b3-b02e71596dd6 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0689647 May 24 11:18:21.974620 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:22.556571 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dfb887f1-64cd-4d2e-b26c-b33ef9c82a72 req-56556d5c-b52b-4b3b-8d65-da2588b8477d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:22.556571 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-dfb887f1-64cd-4d2e-b26c-b33ef9c82a72 req-56556d5c-b52b-4b3b-8d65-da2588b8477d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.0996828 May 24 11:18:22.613995 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-dfb887f1-64cd-4d2e-b26c-b33ef9c82a72 req-06734fba-9a2d-4141-8845-24dee04486a9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0516486 May 24 11:18:23.411953 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-001e13ed-3af2-4137-bffd-c6db0587f7cd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:25.343315 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-91d4ddb4-ce5d-4793-bdda-31e582b03f13 req-80697c31-bd59-49e5-92e9-bdd820317fac tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:25.344837 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-91d4ddb4-ce5d-4793-bdda-31e582b03f13 req-80697c31-bd59-49e5-92e9-bdd820317fac tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.0910976 May 24 11:18:25.406486 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-91d4ddb4-ce5d-4793-bdda-31e582b03f13 req-220eade0-e1e3-4018-aca5-de781366c4c5 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0541763 May 24 11:18:25.749744 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:25.859493 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-085965fc-dff6-4983-8471-cb58a9fdb26a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:26.422649 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 55712) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:26.646451 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dbc6a59d-f3e9-4615-b4b3-7a30e972699f admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:26.673600 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-dbc6a59d-f3e9-4615-b4b3-7a30e972699f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:26.685228 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-19250b68-f175-4df9-8401-61ed19b3ae35 req-cd44ac16-fc00-4c30-a6ed-6406e5729c88 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:26.686754 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-19250b68-f175-4df9-8401-61ed19b3ae35 req-cd44ac16-fc00-4c30-a6ed-6406e5729c88 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1505220 May 24 11:18:26.689593 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-dbc6a59d-f3e9-4615-b4b3-7a30e972699f admin admin] Usage tracker for resource:security_group_rule and tenant:5c61491cad2243aeb097dee44add2098 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:26.694011 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-dbc6a59d-f3e9-4615-b4b3-7a30e972699f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:26.714195 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-dbc6a59d-f3e9-4615-b4b3-7a30e972699f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:26.795213 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-19250b68-f175-4df9-8401-61ed19b3ae35 req-b4728a79-2ffd-473d-bb42-90d99417dd37 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1023247 May 24 11:18:26.887281 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83aa64a8-b73a-4a45-9661-f5a4c35803d8 req-a648f78e-37b8-4553-affe-cc4d9b034225 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:26.888747 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-83aa64a8-b73a-4a45-9661-f5a4c35803d8 req-a648f78e-37b8-4553-affe-cc4d9b034225 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 1099 time: 0.1321764 May 24 11:18:26.926209 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dbc6a59d-f3e9-4615-b4b3-7a30e972699f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:26.936958 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dbc6a59d-f3e9-4615-b4b3-7a30e972699f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:26.949970 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:26.960599 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:26.997299 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-83aa64a8-b73a-4a45-9661-f5a4c35803d8 req-40e5e659-a493-488a-8f21-fb3d93ac6fde service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0956719 May 24 11:18:27.005812 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-dbc6a59d-f3e9-4615-b4b3-7a30e972699f admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=5c61491cad2243aeb097dee44add2098&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5812328 May 24 11:18:27.008722 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 55728) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:27.024477 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dbc6a59d-f3e9-4615-b4b3-7a30e972699f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=11f3d07b-8edd-47c3-b744-addf4d9aa3be,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:27.032561 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-dbc6a59d-f3e9-4615-b4b3-7a30e972699f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:27.094206 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9650459b-3c7e-42d5-ae92-2b4e4cdb032e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:27.132701 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9650459b-3c7e-42d5-ae92-2b4e4cdb032e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:27.204606 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9650459b-3c7e-42d5-ae92-2b4e4cdb032e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:27.212605 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c127b25b-8696-4f27-a058-b6a8c2779865 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:27.232160 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9650459b-3c7e-42d5-ae92-2b4e4cdb032e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=11f3d07b-8edd-47c3-b744-addf4d9aa3be,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:27.236029 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9650459b-3c7e-42d5-ae92-2b4e4cdb032e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:27.237310 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9650459b-3c7e-42d5-ae92-2b4e4cdb032e admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:27.241817 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-9650459b-3c7e-42d5-ae92-2b4e4cdb032e admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/11f3d07b-8edd-47c3-b744-addf4d9aa3be HTTP/1.1" status: 204 len: 168 time: 0.2293386 May 24 11:18:28.016339 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93b47c5e-500a-4984-83a7-afcf10ca16a6 req-45b2724b-5675-4d6d-9e67-5325eb60d152 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:28.017994 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-93b47c5e-500a-4984-83a7-afcf10ca16a6 req-45b2724b-5675-4d6d-9e67-5325eb60d152 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1046224 May 24 11:18:28.080471 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-93b47c5e-500a-4984-83a7-afcf10ca16a6 req-1f0a0d57-890a-4d51-b042-12aec135341a tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0554724 May 24 11:18:28.120620 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 55772) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:28.237929 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-1635425371-network', 'tenant_id': '7c9e587a510c4777ab57643ed31ea73f'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:28.244348 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:28.252694 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Usage tracker for resource:network and tenant:7c9e587a510c4777ab57643ed31ea73f is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:28.256251 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:28.263733 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:28.275875 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Made reservation on behalf of 7c9e587a510c4777ab57643ed31ea73f for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:18:28.278204 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:28.294742 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:28.310316 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:28.314514 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Usage tracker for resource:security_group_rule and tenant:7c9e587a510c4777ab57643ed31ea73f is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:28.317417 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:28.323601 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:28.419360 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:28.423528 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:28.427147 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-03cabd0f-e975-4d4b-803b-eae4689bf452 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:28.456572 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] vlan segment allocate from pool success with {'vlan_id': 514, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:18:28.466482 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=57604184-194b-41df-8615-c443c9e6dfa4,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:28.470285 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:28.472996 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:28.473359 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Added segment f3295bfe-6894-409e-b0cc-96311bd90247 of type vlan for network a21e7e1e-54f7-4844-b98d-f658bdb26a4c May 24 11:18:28.505666 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:28.543329 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:28.545604 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:28.546600 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:28.553674 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:28.663893 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 0.5417407 May 24 11:18:28.666892 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 55786) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:28.670174 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Pushing event updated for resources: {'Network': ['ID=a21e7e1e-54f7-4844-b98d-f658bdb26a4c,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:28.677693 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Request body: {'subnet': {'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-1635425371-subnet', 'tenant_id': '7c9e587a510c4777ab57643ed31ea73f', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:28.681326 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.128s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:28.683547 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:28.684793 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network a21e7e1e-54f7-4844-b98d-f658bdb26a4c details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:18:28.688069 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Usage tracker for resource:subnet and tenant:7c9e587a510c4777ab57643ed31ea73f is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:28.692542 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:28.706697 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:28.719033 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Made reservation on behalf of 7c9e587a510c4777ab57643ed31ea73f for: {'subnet': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:18:28.721948 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-535570794'] for subnet, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:28.994579 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:29.030965 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55796) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:29.197796 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-4b74dfc1-937e-4208-9ad5-3ea2350b720a admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:29.214168 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-4b74dfc1-937e-4208-9ad5-3ea2350b720a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:29.218552 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-4b74dfc1-937e-4208-9ad5-3ea2350b720a admin admin] Usage tracker for resource:security_group_rule and tenant:154bc9a3a0394692bed6678ce048e2d8 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:29.222059 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-4b74dfc1-937e-4208-9ad5-3ea2350b720a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:29.228364 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-4b74dfc1-937e-4208-9ad5-3ea2350b720a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:29.346530 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-534944967', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-535570812'] for subnet, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:29.348608 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-4b74dfc1-937e-4208-9ad5-3ea2350b720a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:29.354786 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:29.366591 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-4b74dfc1-937e-4208-9ad5-3ea2350b720a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:29.377753 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:29.433419 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4b74dfc1-937e-4208-9ad5-3ea2350b720a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b1eae6b0-a4d1-4910-995f-698612cd4788,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:29.442049 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-4b74dfc1-937e-4208-9ad5-3ea2350b720a admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=154bc9a3a0394692bed6678ce048e2d8&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4097073 May 24 11:18:29.443774 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-4b74dfc1-937e-4208-9ad5-3ea2350b720a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.066s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:29.444385 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55808) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:29.446845 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Pushing event updated for resources: {'Subnet': ['ID=1ed78d41-8564-48a2-acc1-2ef21df88d9e,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:29.450092 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.095s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:29.455868 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 55812) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:29.469705 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-da3abda4-a058-4c12-b251-2ef9d1cc1987 req-445e8d8a-d877-4c4c-84d6-623998e397cc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:29.471215 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-da3abda4-a058-4c12-b251-2ef9d1cc1987 req-445e8d8a-d877-4c4c-84d6-623998e397cc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.2163465 May 24 11:18:29.501690 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Checking segment: {'id': 'f3295bfe-6894-409e-b0cc-96311bd90247', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 514, 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:29.502045 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Network hostable DHCP agents. Network: a21e7e1e-54f7-4844-b98d-f658bdb26a4c, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91819) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:18:29.529000 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cd099143-fbe7-46a4-8709-cd774e264f75 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:29.541017 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-da3abda4-a058-4c12-b251-2ef9d1cc1987 req-76c73b53-956e-49cb-89d9-13166906f1df tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0609210 May 24 11:18:29.550160 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Network a21e7e1e-54f7-4844-b98d-f658bdb26a4c is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91819) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:18:29.570043 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cd099143-fbe7-46a4-8709-cd774e264f75 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:29.578713 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, a21e7e1e-54f7-4844-b98d-f658bdb26a4c) {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:29.593131 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-ad9bc10b-b4a0-4018-9d6d-64ae3aeeb444 None None] Network a21e7e1e-54f7-4844-b98d-f658bdb26a4c requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:18:29.602875 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:29.604037 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:29.640723 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cd099143-fbe7-46a4-8709-cd774e264f75 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:29.645148 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 846 time: 0.9758067 May 24 11:18:29.656740 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-34fabc75-a008-4282-9769-ef1ddfe5b428 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:29.684403 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cd099143-fbe7-46a4-8709-cd774e264f75 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b1eae6b0-a4d1-4910-995f-698612cd4788,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:29.690344 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-cd099143-fbe7-46a4-8709-cd774e264f75 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:29.705341 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cd099143-fbe7-46a4-8709-cd774e264f75 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:29.708339 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-cd099143-fbe7-46a4-8709-cd774e264f75 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/b1eae6b0-a4d1-4910-995f-698612cd4788 HTTP/1.1" status: 204 len: 168 time: 0.2624555 May 24 11:18:29.725133 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e61a22c0-cbb5-46f3-9e2f-d19bd9d2f59a admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:29.850786 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e61a22c0-cbb5-46f3-9e2f-d19bd9d2f59a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:29.855914 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e61a22c0-cbb5-46f3-9e2f-d19bd9d2f59a admin admin] Usage tracker for resource:security_group_rule and tenant:2b5b2db411f14c9f875428bf7087132e is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:29.859417 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e61a22c0-cbb5-46f3-9e2f-d19bd9d2f59a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:29.868908 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e61a22c0-cbb5-46f3-9e2f-d19bd9d2f59a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:30.109598 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e61a22c0-cbb5-46f3-9e2f-d19bd9d2f59a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:30.116184 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e61a22c0-cbb5-46f3-9e2f-d19bd9d2f59a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:30.133874 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f88bc2e3-a7ab-4a94-b05b-7360935e8938 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:30.137629 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-29a109a6-78e4-4634-a6ed-e1e8811c8a41 None None] Network a21e7e1e-54f7-4844-b98d-f658bdb26a4c requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:18:30.190875 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e61a22c0-cbb5-46f3-9e2f-d19bd9d2f59a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ccceeb04-02fd-47b4-b8fd-fbed387856aa,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:30.199523 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e61a22c0-cbb5-46f3-9e2f-d19bd9d2f59a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.065s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:30.237230 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e61a22c0-cbb5-46f3-9e2f-d19bd9d2f59a admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=2b5b2db411f14c9f875428bf7087132e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7801116 May 24 11:18:30.237872 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55838) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:30.378075 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-63202e0d-1f0b-487d-8ee2-65458f91c165 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:30.433008 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-63202e0d-1f0b-487d-8ee2-65458f91c165 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:30.506201 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-63202e0d-1f0b-487d-8ee2-65458f91c165 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:30.518700 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fea9e182-90ea-4e53-93c7-26e21a00376d req-1664310a-b802-404f-828c-0425c70a5ec5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:30.520538 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fea9e182-90ea-4e53-93c7-26e21a00376d req-1664310a-b802-404f-828c-0425c70a5ec5 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 1099 time: 0.2299678 May 24 11:18:30.521337 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1266b479-9648-4404-aadb-119bf686fd48 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:30.545766 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-63202e0d-1f0b-487d-8ee2-65458f91c165 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ccceeb04-02fd-47b4-b8fd-fbed387856aa,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:30.560040 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-63202e0d-1f0b-487d-8ee2-65458f91c165 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:30.560923 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-63202e0d-1f0b-487d-8ee2-65458f91c165 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:30.562102 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-63202e0d-1f0b-487d-8ee2-65458f91c165 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/ccceeb04-02fd-47b4-b8fd-fbed387856aa HTTP/1.1" status: 204 len: 168 time: 0.3218682 May 24 11:18:30.580779 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2602baf6-fec0-4adc-8028-c7dfd1e24f99 req-6d645c7a-cd7a-44ec-970d-5cbc7f92e024 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:30.596087 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2602baf6-fec0-4adc-8028-c7dfd1e24f99 req-6d645c7a-cd7a-44ec-970d-5cbc7f92e024 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1844075 May 24 11:18:30.740313 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2602baf6-fec0-4adc-8028-c7dfd1e24f99 req-4b3615ca-676a-4d7a-89e9-202cf2affa04 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1433289 May 24 11:18:30.812174 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fea9e182-90ea-4e53-93c7-26e21a00376d req-eaafad32-f961-44d9-89bd-58780c2086c2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:30.813749 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fea9e182-90ea-4e53-93c7-26e21a00376d req-eaafad32-f961-44d9-89bd-58780c2086c2 service neutron] 10.222.0.93 "GET /v2.0/networks?id=4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 200 len: 918 time: 0.2809298 May 24 11:18:30.914773 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fea9e182-90ea-4e53-93c7-26e21a00376d req-2fb40cd2-e3cb-4885-9b73-68987f8f0aa7 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=02210ade-8c60-41b6-aa82-a104d4fc0447 HTTP/1.1" status: 200 len: 217 time: 0.0852661 May 24 11:18:31.009047 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fea9e182-90ea-4e53-93c7-26e21a00376d req-ef718f88-8058-41a3-b049-2a716c735630 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:31.010608 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fea9e182-90ea-4e53-93c7-26e21a00376d req-ef718f88-8058-41a3-b049-2a716c735630 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=52267672-fd8c-45bd-89ea-d9d02a7b249a HTTP/1.1" status: 200 len: 857 time: 0.0878932 May 24 11:18:31.098087 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fea9e182-90ea-4e53-93c7-26e21a00376d req-c3d816dd-cd88-47a8-bd27-7895b0ce09c5 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=4fec6b68-64f1-4553-8ca5-f86c63ac9714&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0750303 May 24 11:18:31.179951 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55868) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:31.203215 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df822461-dc26-411c-96dc-8da6142346e4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.224892 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-df822461-dc26-411c-96dc-8da6142346e4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:31.229579 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-df822461-dc26-411c-96dc-8da6142346e4 admin admin] Usage tracker for resource:security_group_rule and tenant:6eaa5a0bf29f4b7eb1a411ea030c11f5 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:31.233682 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-df822461-dc26-411c-96dc-8da6142346e4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:31.243828 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-df822461-dc26-411c-96dc-8da6142346e4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:31.246754 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55872) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:31.292631 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fea9e182-90ea-4e53-93c7-26e21a00376d req-9375855e-4ddd-43f3-8b65-57a1434eb9f4 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1833875 May 24 11:18:31.381213 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df822461-dc26-411c-96dc-8da6142346e4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.387457 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-df822461-dc26-411c-96dc-8da6142346e4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.407991 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f7617af6-6707-41a5-bf20-7f7e58a8d90f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:31.516183 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fea9e182-90ea-4e53-93c7-26e21a00376d req-924376dd-fb00-4526-93b4-1b61b0d614b2 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2146227 May 24 11:18:31.546837 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-df822461-dc26-411c-96dc-8da6142346e4 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=6eaa5a0bf29f4b7eb1a411ea030c11f5&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3631043 May 24 11:18:31.546837 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-df822461-dc26-411c-96dc-8da6142346e4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=bd0c34ee-3fba-4379-8c0b-d8bcf10fa740,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:31.550312 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 55880) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:31.551119 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-df822461-dc26-411c-96dc-8da6142346e4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.142s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:31.594924 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Request body: {'network': {'name': 'tempest-TestMinimumBasicScenario-953866096-network', 'tenant_id': '788050c389224d17b163f24bb67902bf'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:31.600607 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:31.605144 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Usage tracker for resource:network and tenant:788050c389224d17b163f24bb67902bf is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:31.608592 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:31.614778 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:31.618018 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-39ac601b-3640-471a-a88d-97114465f699 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.626023 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Made reservation on behalf of 788050c389224d17b163f24bb67902bf for: {'network': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:18:31.628502 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.641691 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.648113 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-39ac601b-3640-471a-a88d-97114465f699 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.659243 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:31.665541 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Usage tracker for resource:security_group_rule and tenant:788050c389224d17b163f24bb67902bf is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:31.668455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:31.674251 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:31.716218 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-39ac601b-3640-471a-a88d-97114465f699 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.719945 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9a093274-4104-4e41-b715-f8654d19db9a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:31.735053 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-39ac601b-3640-471a-a88d-97114465f699 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=bd0c34ee-3fba-4379-8c0b-d8bcf10fa740,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:31.739410 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-39ac601b-3640-471a-a88d-97114465f699 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:31.752652 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-39ac601b-3640-471a-a88d-97114465f699 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.754333 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-39ac601b-3640-471a-a88d-97114465f699 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/bd0c34ee-3fba-4379-8c0b-d8bcf10fa740 HTTP/1.1" status: 204 len: 168 time: 0.2029319 May 24 11:18:31.781020 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.786970 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.795389 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:31.814987 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] vlan segment allocate from pool success with {'vlan_id': 539, 'physical_network': 'physnet1'} {{(pid=91820) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:18:31.839819 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=29c10cc8-55e7-4791-971b-485a28ba237e,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:31.846122 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.846596 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.segments_db [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Added segment 666aa9e5-b138-4841-9cc4-ecbe52ac33c1 of type vlan for network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 May 24 11:18:31.870007 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:31.892178 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.935535 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.937268 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:31.938389 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:31.944517 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-3dbc0b9f-9a97-4f1b-bd68-d0c51fe7f47d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:31.984542 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2fd57a94-afc3-47df-b1b7-986b6ec7dc3a req-226904cd-416c-469a-b47f-acd06dbee445 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:31.986280 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2fd57a94-afc3-47df-b1b7-986b6ec7dc3a req-226904cd-416c-469a-b47f-acd06dbee445 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1139562 May 24 11:18:32.061566 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2fd57a94-afc3-47df-b1b7-986b6ec7dc3a req-375a934c-de98-44f5-9125-432b513fb38f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0670211 May 24 11:18:32.064213 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Pushing event updated for resources: {'Network': ['ID=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:32.067651 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.123s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:32.074227 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 0.8225014 May 24 11:18:32.074707 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55892) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:32.079676 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:18:32.091586 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Request body: {'subnet': {'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestMinimumBasicScenario-953866096-subnet', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'ip_version': 4}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:32.096823 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:32.101538 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Usage tracker for resource:subnet and tenant:788050c389224d17b163f24bb67902bf is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:32.105640 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:32.112508 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:32.135254 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Made reservation on behalf of 788050c389224d17b163f24bb67902bf for: {'subnet': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:18:32.140116 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-535456753'] for subnet, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:32.684223 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71752abf-bbbc-4066-b844-984d90d47734 req-bbb73634-f9bb-41d4-9a54-2e6482076849 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:32.684980 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-71752abf-bbbc-4066-b844-984d90d47734 req-bbb73634-f9bb-41d4-9a54-2e6482076849 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Synchronizing usage tracker for tenant:7c9e587a510c4777ab57643ed31ea73f on resource:network {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:18:32.708708 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-71752abf-bbbc-4066-b844-984d90d47734 req-bbb73634-f9bb-41d4-9a54-2e6482076849 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Unset dirty status for tenant:7c9e587a510c4777ab57643ed31ea73f on resource:network {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:18:32.713771 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-bbb73634-f9bb-41d4-9a54-2e6482076849 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/networks?id=a21e7e1e-54f7-4844-b98d-f658bdb26a4c HTTP/1.1" status: 200 len: 809 time: 0.2555931 May 24 11:18:32.729853 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-fb37fc70-52e6-4cb1-862c-3a60e10e90c9 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/quotas/7c9e587a510c4777ab57643ed31ea73f HTTP/1.1" status: 200 len: 375 time: 0.0134890 May 24 11:18:32.744295 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-534788555', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-535456743'] for subnet, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:32.749309 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:32.784667 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-ac4a4b29-e6ce-490c-9854-28e49342388f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=7c9e587a510c4777ab57643ed31ea73f&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0488801 May 24 11:18:32.830605 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Pushing event updated for resources: {'Subnet': ['ID=6a80c579-20d5-46c7-8c07-6438f1367afd,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:32.839144 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:32.839889 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Checking segment: {'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:32.840287 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Network hostable DHCP agents. Network: 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91820) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:18:32.869921 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91820) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:18:32.885395 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397) {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:32.897329 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:32.898033 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:32.898398 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-0739bfe9-61fe-44ce-be47-b30acca4e98a None None] Network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:18:32.933956 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-13dae5ac-3a75-49b3-b2c4-846dd72826d9 admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 850 time: 0.8580735 May 24 11:18:32.937252 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 55930) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:32.947048 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Request body: {'router': {'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:32.955355 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:32.962523 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Usage tracker for resource:router and tenant:788050c389224d17b163f24bb67902bf is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:32.968021 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:32.981770 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:32.987132 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-7f1268d7-3dbb-4535-903b-a06e8f0c2916 service neutron] 10.222.0.18 "GET /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1919601 May 24 11:18:32.998868 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-684314', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4157'] for router, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:33.000111 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-684570', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-687045', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-715560', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4164'] for router, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:33.020131 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:33.165153 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-ca44a244-7b08-42b0-a7dc-30bacf36de56 service neutron] 10.222.0.18 "GET /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1696837 May 24 11:18:33.186917 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpea482e27-f15a-5756-a111-7e0dba035002-0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'fixed_ips': [{'subnet_id': '6a80c579-20d5-46c7-8c07-6438f1367afd'}]}} from dv-cinder-784092-9-iscsi. {{(pid=91821) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 11:18:33.189400 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:33.419144 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.db_base_plugin_common [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Allocated IP 172.20.0.2 (0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397/6a80c579-20d5-46c7-8c07-6438f1367afd/f5a705e5-e33a-4169-b3ca-2428f9726466) {{(pid=91821) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:18:33.423050 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-687072', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-684328', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4178'] for router, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:33.462274 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b262857-4c52-4c6f-8b9c-4fa18cf85e48 req-7ce02ff2-6013-46ec-8e93-991e6086530c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:33.466302 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9b262857-4c52-4c6f-8b9c-4fa18cf85e48 req-7ce02ff2-6013-46ec-8e93-991e6086530c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1966007 May 24 11:18:33.601059 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9b262857-4c52-4c6f-8b9c-4fa18cf85e48 req-3fd0c57b-e11d-4856-b3e5-1274e99ecdfd tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1068957 May 24 11:18:33.646987 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7b03882c-f475-4551-bb8b-58ca56361ae3 req-cc0db3e7-cddc-4bbf-a010-8be060bf78df tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 210 time: 0.2094562 May 24 11:18:33.833491 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Notify callbacks [] for port, precommit_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:33.851076 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Transition to ACTIVE for port object f5a705e5-e33a-4169-b3ca-2428f9726466 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:33.902590 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Transition to ACTIVE for port object f5a705e5-e33a-4169-b3ca-2428f9726466 will not be triggered until provisioned by entity DHCP. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:33.913702 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:33.954107 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:34.137980 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 is already hosted by enough agents. {{(pid=91821) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:34.139288 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.base_scheduler [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Resources bound (agent type, host, resource id): {{(pid=91821) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:34.157778 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Received port after_create {{(pid=91821) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:18:34.208584 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:34.252803 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:34.326755 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Attempting to bind port f5a705e5-e33a-4169-b3ca-2428f9726466 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:18:34.327239 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Attempting to bind port f5a705e5-e33a-4169-b3ca-2428f9726466 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:18:34.327992 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Attempting to bind port f5a705e5-e33a-4169-b3ca-2428f9726466 on network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:34.342333 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Pushing event updated for resources: {'Port': ['ID=f5a705e5-e33a-4169-b3ca-2428f9726466,revision_number=1']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:34.373116 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:34.374945 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:34.375392 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:34.375824 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:34.380515 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:34.380995 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:34.381468 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:34.381905 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:34.382526 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:34.384970 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:34.385451 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:34.386585 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:34.402212 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:18:34.402717 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Checking segment: {'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:34.403245 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:34.403744 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Bound using segment: {'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:18:34.404345 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Bound port: f5a705e5-e33a-4169-b3ca-2428f9726466, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:18:34.405475 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:34.441128 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.477s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:34.455897 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Allocated IP 172.27.0.200 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/54c19980-7452-40ac-b0f5-c89b800a3d24) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:18:34.576539 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] For port f5a705e5-e33a-4169-b3ca-2428f9726466, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:18:34.581051 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] For port f5a705e5-e33a-4169-b3ca-2428f9726466, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=f5a705e5-e33a-4169-b3ca-2428f9726466,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:18:34.589358 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:34.609592 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-1969920a-de31-4b65-a8f3-d82eb3bfd2c9 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:34.657440 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:34.694165 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:34.700922 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:34.798809 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Pushing event updated for resources: {'Port': ['ID=f5a705e5-e33a-4169-b3ca-2428f9726466,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:34.806877 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.197s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:34.819047 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Pushing event updated for resources: {'Port': ['ID=54c19980-7452-40ac-b0f5-c89b800a3d24,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:34.826985 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:34.829852 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:34.831244 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:34.842114 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:18:35.341842 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-17682bef-5639-4f3d-8dc9-9352f35bbbc4 req-5f333cbb-7b8a-466b-ac7c-f74ff220b5bd tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 210 time: 0.0645194 May 24 11:18:35.350041 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2822f5ee-686f-4bda-a61c-3c12557c28db req-95f08038-a077-47e3-82cd-b2c594e63dad tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:35.352959 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2822f5ee-686f-4bda-a61c-3c12557c28db req-95f08038-a077-47e3-82cd-b2c594e63dad tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.4561388 May 24 11:18:35.557755 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2822f5ee-686f-4bda-a61c-3c12557c28db req-1ba790c9-64a5-4d57-afee-30c63ab632b8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1886942 May 24 11:18:35.616359 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71752abf-bbbc-4066-b844-984d90d47734 req-56b73b18-973f-4ef8-9dac-fbe48cdca7e8 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:35.618302 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-56b73b18-973f-4ef8-9dac-fbe48cdca7e8 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.93 "GET /v2.0/networks?id=a21e7e1e-54f7-4844-b98d-f658bdb26a4c HTTP/1.1" status: 200 len: 809 time: 0.1873608 May 24 11:18:35.641134 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:35.642498 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Request body: {'port': {'device_id': '78ccab16-0ff0-4b84-8d91-71e4092110e9', 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c', 'admin_state_up': True, 'tenant_id': '7c9e587a510c4777ab57643ed31ea73f'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:35.860867 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:35.866297 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Usage tracker for resource:port and tenant:7c9e587a510c4777ab57643ed31ea73f is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:35.871860 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Quota usage for port was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:35.883569 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:35.896573 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Made reservation on behalf of 7c9e587a510c4777ab57643ed31ea73f for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:18:35.901352 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:36.148145 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Allocated IP 172.20.0.11 (a21e7e1e-54f7-4844-b98d-f658bdb26a4c/1ed78d41-8564-48a2-acc1-2ef21df88d9e/ff3663f5-a200-4bed-988a-f47af0d746d7) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:18:36.161256 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-26e4d3bf-512c-4142-a32b-ec6b76b68d7b None None] Network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:18:36.252414 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-684307', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-687011'] for router, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:36.277213 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-40b785ff-0979-42e4-8130-1eb1133dd837 None None] Provisioning for port f5a705e5-e33a-4169-b3ca-2428f9726466 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:36.648662 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-344a92be-ebfa-4cd1-91dc-7a19ea9e976b req-16e6c630-2e45-4865-b956-eedc3a94de9a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 210 time: 0.0952613 May 24 11:18:36.692850 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:36.694150 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Router 684315b8-ba5e-4d23-8963-64aca3b067cb is scheduled to L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 with binding_index 1 {{(pid=91820) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 11:18:36.748000 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:36.754502 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-79a9f8e0-3486-418b-9789-3ecb8cd53e2e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:36.839122 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Network a21e7e1e-54f7-4844-b98d-f658bdb26a4c is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:36.844582 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:36.849970 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:18:36.938732 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Pushing event updated for resources: {'Port': ['ID=ff3663f5-a200-4bed-988a-f47af0d746d7,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:36.944189 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.189s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:36.952897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:36.953623 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:36.996364 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.3583624 May 24 11:18:37.011069 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Request body: {'port': {'device_id': '78ccab16-0ff0-4b84-8d91-71e4092110e9', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:37.013290 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:18:37.025983 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Sync routers for ids ['684315b8-ba5e-4d23-8963-64aca3b067cb'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:18:37.048995 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Notify callbacks [] for router, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:37.050987 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] 10.222.0.18 "POST /v2.0/routers HTTP/1.1" status: 201 len: 926 time: 4.1116242 May 24 11:18:37.054712 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 56036) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:37.067300 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device f5a705e5-e33a-4169-b3ca-2428f9726466 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:37.074107 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Routers {'684315b8-ba5e-4d23-8963-64aca3b067cb'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:18:37.075174 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['684315b8-ba5e-4d23-8963-64aca3b067cb'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:37.170991 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:37.268437 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port f5a705e5-e33a-4169-b3ca-2428f9726466 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:37.275312 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port f5a705e5-e33a-4169-b3ca-2428f9726466 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:37.275695 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:37.340131 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Set router 684315b8-ba5e-4d23-8963-64aca3b067cb gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:18:37.451920 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:37.508867 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port f5a705e5-e33a-4169-b3ca-2428f9726466, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=f5a705e5-e33a-4169-b3ca-2428f9726466,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:37.513612 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:37.522594 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c9771ac0-1c52-4756-978f-810940b6daed None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:37.646738 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=f5a705e5-e33a-4169-b3ca-2428f9726466,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:37.666828 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.144s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:37.714817 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id ff3663f5-a200-4bed-988a-f47af0d746d7 {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:18:37.724212 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:37.744212 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Transition to ACTIVE for port object ff3663f5-a200-4bed-988a-f47af0d746d7 will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:37.750757 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:37.760812 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:37.975274 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Pushing event updated for resources: {'Port': ['ID=ff3663f5-a200-4bed-988a-f47af0d746d7,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:37.981627 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.221s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:37.983905 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:18:35Z', 'revision_number': 3, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:34Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:37.994399 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Checking router: 684315b8-ba5e-4d23-8963-64aca3b067cb for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:18:37.994755 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Updating router 684315b8-ba5e-4d23-8963-64aca3b067cb port 54c19980-7452-40ac-b0f5-c89b800a3d24 binding host dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:18:38.030691 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Attempting to bind port ff3663f5-a200-4bed-988a-f47af0d746d7 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:18:38.031143 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Attempting to bind port ff3663f5-a200-4bed-988a-f47af0d746d7 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': 'f3295bfe-6894-409e-b0cc-96311bd90247', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 514, 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:18:38.031414 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Attempting to bind port ff3663f5-a200-4bed-988a-f47af0d746d7 on network a21e7e1e-54f7-4844-b98d-f658bdb26a4c {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:38.036543 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Port ff3663f5-a200-4bed-988a-f47af0d746d7 on network a21e7e1e-54f7-4844-b98d-f658bdb26a4c not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:18:38.036859 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Attempting to bind port ff3663f5-a200-4bed-988a-f47af0d746d7 on network a21e7e1e-54f7-4844-b98d-f658bdb26a4c {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:38.045303 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 22), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:18:38.045788 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Checking segment: {'id': 'f3295bfe-6894-409e-b0cc-96311bd90247', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 514, 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:38.046303 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Bound using segment: {'id': 'f3295bfe-6894-409e-b0cc-96311bd90247', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 514, 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:18:38.046856 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Bound port: ff3663f5-a200-4bed-988a-f47af0d746d7, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'f3295bfe-6894-409e-b0cc-96311bd90247', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 514, 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:18:38.047357 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:38.070811 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-73574114-ef75-4c01-aa82-524bc5b479b1 req-cc400019-5c70-438b-a02b-0a3e82bed955 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:38.073230 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-73574114-ef75-4c01-aa82-524bc5b479b1 req-cc400019-5c70-438b-a02b-0a3e82bed955 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.3181825 May 24 11:18:38.127805 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:38.166457 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-73574114-ef75-4c01-aa82-524bc5b479b1 req-8a0212ce-6dfc-4003-baca-1d420fb466aa tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0864739 May 24 11:18:38.389520 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] For port ff3663f5-a200-4bed-988a-f47af0d746d7, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:18:38.395204 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] For port ff3663f5-a200-4bed-988a-f47af0d746d7, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=ff3663f5-a200-4bed-988a-f47af0d746d7,segment=NetworkSegment(f3295bfe-6894-409e-b0cc-96311bd90247),segment_id=f3295bfe-6894-409e-b0cc-96311bd90247)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:18:38.404895 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:38.420608 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e99beb93-0d23-4000-803e-d65d0e2cf3fa tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:38.511828 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:38.592512 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Pushing event updated for resources: {'Port': ['ID=ff3663f5-a200-4bed-988a-f47af0d746d7,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:38.616969 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.196s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:38.671837 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:38.672434 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:38.673874 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] 10.222.0.93 "PUT /v2.0/ports/ff3663f5-a200-4bed-988a-f47af0d746d7 HTTP/1.1" status: 200 len: 1094 time: 1.6694958 May 24 11:18:38.674384 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '78ccab16-0ff0-4b84-8d91-71e4092110e9', 'tag': 'ff3663f5-a200-4bed-988a-f47af0d746d7'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:18:38.736657 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c6049fc8-c203-4e0a-8c4a-b0e97246718c req-5d84fd4e-abba-479f-b5c6-1829848c3999 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:38.738875 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c6049fc8-c203-4e0a-8c4a-b0e97246718c req-5d84fd4e-abba-479f-b5c6-1829848c3999 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 944 time: 0.2996259 May 24 11:18:38.834648 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Notify callbacks [] for port, precommit_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:38.854126 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.provisioning_blocks [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Transition to ACTIVE for port object 54c19980-7452-40ac-b0f5-c89b800a3d24 will not be triggered until provisioned by entity L2. {{(pid=91822) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:38.864604 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:38.870185 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c6049fc8-c203-4e0a-8c4a-b0e97246718c req-e6268621-709f-41c0-95cf-466e10f58c05 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0893669 May 24 11:18:38.882155 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-099a4a1a-0c7d-4e78-b27a-03391453046d None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.002s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:38.964121 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device f5a705e5-e33a-4169-b3ca-2428f9726466 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:38.975879 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Allocated IP 172.20.0.1 (0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397/6a80c579-20d5-46c7-8c07-6438f1367afd/90af9233-b13f-40e5-a0f8-872fe40ddfed) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:18:38.978681 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56084) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:39.026032 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Pushing event updated for resources: {'Port': ['ID=54c19980-7452-40ac-b0f5-c89b800a3d24,revision_number=2']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:39.033255 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-0f6eda02-d30b-42c8-9741-a96e5990d35e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:39.042494 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-0f6eda02-d30b-42c8-9741-a96e5990d35e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/networks?id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add HTTP/1.1" status: 200 len: 814 time: 0.5400388 May 24 11:18:39.065055 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-8d0734d4-36f0-43f8-a80c-e5a26bf5b690 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/quotas/003eb49c73ae4e6a962cbfb425b3c1eb HTTP/1.1" status: 200 len: 375 time: 0.0201237 May 24 11:18:39.079262 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Attempting to bind port 54c19980-7452-40ac-b0f5-c89b800a3d24 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:18:39.080257 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Attempting to bind port 54c19980-7452-40ac-b0f5-c89b800a3d24 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:18:39.080806 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Attempting to bind port 54c19980-7452-40ac-b0f5-c89b800a3d24 on network 858b8fad-a73f-4578-9079-6e731b9ff7e2 {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:39.089506 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:39.090147 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:39.090900 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:39.091922 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:39.092946 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:39.093345 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:39.093694 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:39.094070 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:39.094432 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:39.094906 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:39.095300 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:39.095623 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:18:39.101947 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.219s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:39.106153 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:18:39.106292 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Checking segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91822) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:39.106637 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:39.107206 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Bound using segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:18:39.107700 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Bound port: 54c19980-7452-40ac-b0f5-c89b800a3d24, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:18:39.108085 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:39.155980 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71752abf-bbbc-4066-b844-984d90d47734 req-8b6b8f1f-33b7-41d1-9ee2-094ba15ad1eb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:39.159153 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-8b6b8f1f-33b7-41d1-9ee2-094ba15ad1eb service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=7c9e587a510c4777ab57643ed31ea73f&device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 1097 time: 0.4175141 May 24 11:18:39.167433 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port f5a705e5-e33a-4169-b3ca-2428f9726466 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:39.167935 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:39.225988 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] For port 54c19980-7452-40ac-b0f5-c89b800a3d24, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91822) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:18:39.230070 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] For port 54c19980-7452-40ac-b0f5-c89b800a3d24, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=54c19980-7452-40ac-b0f5-c89b800a3d24,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91822) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:18:39.236061 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:39.246076 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:39.371602 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Routers {'684315b8-ba5e-4d23-8963-64aca3b067cb'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:18:39.372250 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['684315b8-ba5e-4d23-8963-64aca3b067cb'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:39.396879 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-aebfc215-0399-4083-8e40-dcc5bad971e0 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=ff3663f5-a200-4bed-988a-f47af0d746d7 HTTP/1.1" status: 200 len: 217 time: 0.2255077 May 24 11:18:39.400252 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d34c53e4-13fa-4305-a86a-fc3c108d7e70 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:39.472928 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-d34c53e4-13fa-4305-a86a-fc3c108d7e70 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:39.494012 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-33d8d743-1c30-4d5c-a5e9-70d28179583e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&fields=id HTTP/1.1" status: 200 len: 348 time: 0.4275978 May 24 11:18:39.522056 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-d34c53e4-13fa-4305-a86a-fc3c108d7e70 admin admin] Usage tracker for resource:security_group_rule and tenant:aca8f1828e1943da8f3c653810e91c05 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:39.545545 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:39.598594 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-d34c53e4-13fa-4305-a86a-fc3c108d7e70 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:39.608135 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Transition to ACTIVE for port object 90af9233-b13f-40e5-a0f8-872fe40ddfed will not be triggered until provisioned by entity DHCP. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:39.611624 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-d34c53e4-13fa-4305-a86a-fc3c108d7e70 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:39.627923 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71752abf-bbbc-4066-b844-984d90d47734 req-59c2b338-207d-4c90-85e8-4c606190dc6a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:39.631370 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-59c2b338-207d-4c90-85e8-4c606190dc6a service neutron] 10.222.0.93 "GET /v2.0/subnets?id=1ed78d41-8564-48a2-acc1-2ef21df88d9e HTTP/1.1" status: 200 len: 849 time: 0.2182403 May 24 11:18:39.665937 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:39.675249 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:39.696493 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-84c88bef-8297-4c0f-8c4c-e5ad219d32f9 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=a21e7e1e-54f7-4844-b98d-f658bdb26a4c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0461030 May 24 11:18:39.736566 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Pushing event updated for resources: {'Port': ['ID=54c19980-7452-40ac-b0f5-c89b800a3d24,revision_number=3']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:39.741397 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.495s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:39.763988 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] Set router 684315b8-ba5e-4d23-8963-64aca3b067cb gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:18:39.778836 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:39.782922 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:39.785106 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Pushing event updated for resources: {'Port': ['ID=90af9233-b13f-40e5-a0f8-872fe40ddfed,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:39.787589 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:18:39.821978 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.147s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:39.860456 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-687570'] for router_interface, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:39.935594 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '78ccab16-0ff0-4b84-8d91-71e4092110e9', 'tag': 'ff3663f5-a200-4bed-988a-f47af0d746d7', 'status': 'completed', 'code': 200} May 24 11:18:40.067911 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:40.069312 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-91b1ce87-092a-44ca-b89d-09f2e88a765e service neutron] 10.222.0.18 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.5739756 May 24 11:18:40.170029 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d34c53e4-13fa-4305-a86a-fc3c108d7e70 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:40.179495 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-5961cb85-74c6-45be-99e6-481bc0cc7987 None None] Provisioning for port 90af9233-b13f-40e5-a0f8-872fe40ddfed completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:40.185843 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-5961cb85-74c6-45be-99e6-481bc0cc7987 None None] Provisioning complete for port 90af9233-b13f-40e5-a0f8-872fe40ddfed triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:40.186613 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-5961cb85-74c6-45be-99e6-481bc0cc7987 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:40.248586 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d34c53e4-13fa-4305-a86a-fc3c108d7e70 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:40.258114 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-5961cb85-74c6-45be-99e6-481bc0cc7987 None None] Port 90af9233-b13f-40e5-a0f8-872fe40ddfed cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:18:40.278368 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-98c7ac23-e045-4030-9d2f-36b238461770 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:40.319290 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3d1b3dbe-1d29-4a3b-82ec-c856a31291a6 req-3c30d107-bdc5-4567-b84c-d7b68fc312fd tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:40.320967 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3d1b3dbe-1d29-4a3b-82ec-c856a31291a6 req-3c30d107-bdc5-4567-b84c-d7b68fc312fd tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 944 time: 0.2345109 May 24 11:18:40.374320 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-d34c53e4-13fa-4305-a86a-fc3c108d7e70 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=aca8f1828e1943da8f3c653810e91c05&name=default HTTP/1.1" status: 200 len: 2979 time: 1.3817019 May 24 11:18:40.374320 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-7af82989-823c-4b14-809c-f8b47dcbbdba service neutron] 10.222.0.93 "GET /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.6617095 May 24 11:18:40.378835 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56102) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:40.401536 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d34c53e4-13fa-4305-a86a-fc3c108d7e70 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=3a44de3c-6af7-4f34-bd6a-df763ea9816a,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:40.421603 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3d1b3dbe-1d29-4a3b-82ec-c856a31291a6 req-d8566676-8e81-4093-85ed-73b71f838185 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0923092 May 24 11:18:40.440436 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d34c53e4-13fa-4305-a86a-fc3c108d7e70 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.165s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:40.466577 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:18:35Z', 'revision_number': 3, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:39Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:40.477005 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:40.485080 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:40.486477 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-240c845f-e3c1-456c-a1cf-a2632273e6e2 None None] The sync data for ids ['684315b8-ba5e-4d23-8963-64aca3b067cb'] in dv-cinder-784092-9-iscsi is: [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:18:35Z', 'revision_number': 3, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:39Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:18:40.516087 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:40.537468 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-bb6f3e20-7885-42fa-b201-f187c97ef517 service neutron] 10.222.0.18 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3994093 May 24 11:18:40.607683 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5119a5c1-bfd5-463d-b6fe-560160e74cb4 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:40.620521 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Pushing event updated for resources: {'Port': ['ID=90af9233-b13f-40e5-a0f8-872fe40ddfed,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:40.625943 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.110s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:40.629974 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-533332', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-687487'] for router_interface, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:40.679216 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71752abf-bbbc-4066-b844-984d90d47734 req-3cf4aa53-60c1-47b7-9ba9-7f000994f83a service neutron] 10.222.0.93 "GET /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2794597 May 24 11:18:40.691116 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5119a5c1-bfd5-463d-b6fe-560160e74cb4 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:40.908405 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-36c2af9d-ce95-496c-b4fb-221090f3021b req-e20cd8d7-fe51-4328-86f1-d99337ae6eff tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 210 time: 0.0921068 May 24 11:18:40.931175 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:40.937732 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:40.961746 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5119a5c1-bfd5-463d-b6fe-560160e74cb4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:40.969339 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-043b5f4f-2ea5-472b-8933-8ff3ef6ae76b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:40.996966 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0f9e4b4b-4ce9-4ff7-a5c0-fcf2896a5dfb req-78cb107f-3407-4ff2-bf3a-83c2affd44cd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:40.998607 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0f9e4b4b-4ce9-4ff7-a5c0-fcf2896a5dfb req-78cb107f-3407-4ff2-bf3a-83c2affd44cd service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=7c9e587a510c4777ab57643ed31ea73f&device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 1097 time: 0.2394147 May 24 11:18:40.999503 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5119a5c1-bfd5-463d-b6fe-560160e74cb4 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=3a44de3c-6af7-4f34-bd6a-df763ea9816a,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:41.002362 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5119a5c1-bfd5-463d-b6fe-560160e74cb4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:41.012256 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5119a5c1-bfd5-463d-b6fe-560160e74cb4 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:41.013481 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-5119a5c1-bfd5-463d-b6fe-560160e74cb4 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/3a44de3c-6af7-4f34-bd6a-df763ea9816a HTTP/1.1" status: 204 len: 168 time: 0.6337116 May 24 11:18:41.087268 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 54c19980-7452-40ac-b0f5-c89b800a3d24 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:41.317776 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 54c19980-7452-40ac-b0f5-c89b800a3d24 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:41.331084 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 54c19980-7452-40ac-b0f5-c89b800a3d24 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:41.331479 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:41.436255 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-0e44088e-2c0c-46c8-9762-b7686230fab3 None None] Provisioning complete for port 90af9233-b13f-40e5-a0f8-872fe40ddfed triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:41.436425 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-0e44088e-2c0c-46c8-9762-b7686230fab3 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:41.505191 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Router 684315b8-ba5e-4d23-8963-64aca3b067cb has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:18:41.588543 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-0e44088e-2c0c-46c8-9762-b7686230fab3 None None] Port 90af9233-b13f-40e5-a0f8-872fe40ddfed cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:18:41.597001 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0f9e4b4b-4ce9-4ff7-a5c0-fcf2896a5dfb req-87cefd88-14a3-462f-8e53-37a0813e6fa1 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:41.601231 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0f9e4b4b-4ce9-4ff7-a5c0-fcf2896a5dfb req-87cefd88-14a3-462f-8e53-37a0813e6fa1 service neutron] 10.222.0.93 "GET /v2.0/networks?id=a21e7e1e-54f7-4844-b98d-f658bdb26a4c HTTP/1.1" status: 200 len: 910 time: 0.5897975 May 24 11:18:41.648454 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:41.672285 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 54c19980-7452-40ac-b0f5-c89b800a3d24, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=54c19980-7452-40ac-b0f5-c89b800a3d24,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:41.680733 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:41.683288 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:41.768849 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=54c19980-7452-40ac-b0f5-c89b800a3d24,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:41.775688 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:41.798010 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 56126) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:41.811506 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0f9e4b4b-4ce9-4ff7-a5c0-fcf2896a5dfb req-1f5c30ce-b0be-45ee-b5fc-1521dfb8f41e service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=ff3663f5-a200-4bed-988a-f47af0d746d7 HTTP/1.1" status: 200 len: 217 time: 0.1983883 May 24 11:18:41.829760 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-185c34e5-068c-499d-9a07-ef54406cb1b8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:41.890739 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:18:41.906914 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-185c34e5-068c-499d-9a07-ef54406cb1b8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:41.923085 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] 10.222.0.18 "PUT /v2.0/routers/684315b8-ba5e-4d23-8963-64aca3b067cb/add_router_interface HTTP/1.1" status: 200 len: 503 time: 4.8488154 May 24 11:18:41.923085 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-185c34e5-068c-499d-9a07-ef54406cb1b8 admin admin] Usage tracker for resource:security_group_rule and tenant:4e5af322213540ac8bc4839dbf647227 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:41.937389 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-185c34e5-068c-499d-9a07-ef54406cb1b8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:41.946730 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0f9e4b4b-4ce9-4ff7-a5c0-fcf2896a5dfb req-05f10485-244d-4200-a6d3-aa888eecc4c2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:41.948802 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0f9e4b4b-4ce9-4ff7-a5c0-fcf2896a5dfb req-05f10485-244d-4200-a6d3-aa888eecc4c2 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=1ed78d41-8564-48a2-acc1-2ef21df88d9e HTTP/1.1" status: 200 len: 849 time: 0.1197195 May 24 11:18:41.951261 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-185c34e5-068c-499d-9a07-ef54406cb1b8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:42.043483 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0f9e4b4b-4ce9-4ff7-a5c0-fcf2896a5dfb req-a734e7a4-1481-4227-96ac-f273b94105f3 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=a21e7e1e-54f7-4844-b98d-f658bdb26a4c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0841503 May 24 11:18:42.057495 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c31b0c2c-2646-4e75-8bb1-b61fdd5cd333 req-42511827-4fd7-4018-bd3d-c2751c0b37e0 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:42.058995 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-c31b0c2c-2646-4e75-8bb1-b61fdd5cd333 req-42511827-4fd7-4018-bd3d-c2751c0b37e0 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Synchronizing usage tracker for tenant:7c9e587a510c4777ab57643ed31ea73f on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:18:42.085319 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-185c34e5-068c-499d-9a07-ef54406cb1b8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:42.093222 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-185c34e5-068c-499d-9a07-ef54406cb1b8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:42.180338 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-96065557-498a-401c-85df-1707e6a1474b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:42.182766 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-c31b0c2c-2646-4e75-8bb1-b61fdd5cd333 req-42511827-4fd7-4018-bd3d-c2751c0b37e0 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Unset dirty status for tenant:7c9e587a510c4777ab57643ed31ea73f on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:18:42.184076 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c31b0c2c-2646-4e75-8bb1-b61fdd5cd333 req-42511827-4fd7-4018-bd3d-c2751c0b37e0 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 944 time: 0.4638715 May 24 11:18:42.246225 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0c736527-4bf5-4da4-980b-2c46ef1ffbb3 req-98722ede-c664-46ce-8ee5-c3ac6adfc92b tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 210 time: 0.1442862 May 24 11:18:42.255625 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0f9e4b4b-4ce9-4ff7-a5c0-fcf2896a5dfb req-d40a1fe0-a4c2-43fc-90cd-87490087a4f1 service neutron] 10.222.0.93 "GET /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1919935 May 24 11:18:42.280263 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-c31b0c2c-2646-4e75-8bb1-b61fdd5cd333 req-464797cb-95b7-472e-a053-2d9d2d2f2aaf tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Synchronizing usage tracker for tenant:7c9e587a510c4777ab57643ed31ea73f on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:18:42.281508 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-185c34e5-068c-499d-9a07-ef54406cb1b8 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=4e5af322213540ac8bc4839dbf647227&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4767818 May 24 11:18:42.281508 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 56142) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:42.298797 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-185c34e5-068c-499d-9a07-ef54406cb1b8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a63ecabf-3c41-46a6-960f-5aaa56063bfe,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:42.302715 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-185c34e5-068c-499d-9a07-ef54406cb1b8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.122s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:42.363612 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ac23e5ce-6147-4227-a154-0ce403c5a19f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:42.396155 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ac23e5ce-6147-4227-a154-0ce403c5a19f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:42.470772 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ac23e5ce-6147-4227-a154-0ce403c5a19f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:42.477812 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-cd099143-fbe7-46a4-8709-cd774e264f75 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:42.483551 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0f9e4b4b-4ce9-4ff7-a5c0-fcf2896a5dfb req-741be268-d2d6-4107-89ab-c7d7226b5fd1 service neutron] 10.222.0.93 "GET /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2178671 May 24 11:18:42.486205 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-c31b0c2c-2646-4e75-8bb1-b61fdd5cd333 req-464797cb-95b7-472e-a053-2d9d2d2f2aaf tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Unset dirty status for tenant:7c9e587a510c4777ab57643ed31ea73f on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:18:42.487737 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c31b0c2c-2646-4e75-8bb1-b61fdd5cd333 req-464797cb-95b7-472e-a053-2d9d2d2f2aaf tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2860789 May 24 11:18:42.593929 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac23e5ce-6147-4227-a154-0ce403c5a19f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a63ecabf-3c41-46a6-960f-5aaa56063bfe,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:42.595874 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ac23e5ce-6147-4227-a154-0ce403c5a19f admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:42.599909 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-ac23e5ce-6147-4227-a154-0ce403c5a19f admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/a63ecabf-3c41-46a6-960f-5aaa56063bfe HTTP/1.1" status: 204 len: 168 time: 0.3173714 May 24 11:18:42.617365 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ac23e5ce-6147-4227-a154-0ce403c5a19f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.139s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:42.714100 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c439d947-68ba-4b75-90a9-1a32e0388f67 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:42.715608 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c439d947-68ba-4b75-90a9-1a32e0388f67 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "GET /v2.0/networks?id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add HTTP/1.1" status: 200 len: 814 time: 0.2291169 May 24 11:18:42.735867 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Request body: {'port': {'device_id': '9233a97a-5cf6-4529-8a81-39073d46adae', 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add', 'admin_state_up': True, 'tenant_id': '003eb49c73ae4e6a962cbfb425b3c1eb'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:42.744686 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56156) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:42.754503 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Request body: {'network': {'name': 'tempest-TestMinimumBasicScenario-648060998-network', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:42.773399 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:42.787121 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Usage tracker for resource:network and tenant:675d4e36e7c14b46864dd445f05bc0ec is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:42.796472 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:42.807416 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:42.827583 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Made reservation on behalf of 675d4e36e7c14b46864dd445f05bc0ec for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:18:42.830552 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:42.859217 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:42.877753 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device ff3663f5-a200-4bed-988a-f47af0d746d7 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:18:42.912915 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:42.923256 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:42.929132 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Usage tracker for resource:security_group_rule and tenant:675d4e36e7c14b46864dd445f05bc0ec is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:42.932441 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:42.935100 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:42.949573 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:42.954669 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Made reservation on behalf of 003eb49c73ae4e6a962cbfb425b3c1eb for: {'port': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:18:42.959321 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:43.023158 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 54c19980-7452-40ac-b0f5-c89b800a3d24 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:43.088242 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:43.094691 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:43.098571 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-72a553be-ff04-49a1-b3d8-e15f10aa7ac1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:43.144863 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Sync routers for ids ['684315b8-ba5e-4d23-8963-64aca3b067cb'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:18:43.148272 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] vlan segment allocate from pool success with {'vlan_id': 531, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:18:43.155767 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b1c18bf3-361f-4b94-a982-45ecca0f10fd,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:43.161345 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:43.165807 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:43.166075 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Added segment 88ac69f8-b455-4003-825b-2df6e05e49fa of type vlan for network 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a May 24 11:18:43.182821 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Routers {'684315b8-ba5e-4d23-8963-64aca3b067cb'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:18:43.183546 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['684315b8-ba5e-4d23-8963-64aca3b067cb'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:43.207127 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:43.261189 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:43.263530 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:43.264302 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:43.276113 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4f992265-9493-421d-8304-b8324d7702c6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:43.298632 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Allocated IP 172.20.0.10 (bb3e0f6d-2046-4828-8f43-5f99ba5f3add/1107b450-6c27-4f13-8b6e-abdbbbb1ea0f/bb147152-8c96-4e6d-b982-2e6076cc6145) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:18:43.462379 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 0.7154408 May 24 11:18:43.467485 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 54c19980-7452-40ac-b0f5-c89b800a3d24 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:43.468078 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:43.468296 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Pushing event updated for resources: {'Network': ['ID=9672fa3f-743d-4d68-8dd2-0bf684ca5e6a,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:43.470401 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56180) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:43.473875 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.198s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:43.477718 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Request body: {'subnet': {'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestMinimumBasicScenario-648060998-subnet', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'ip_version': 4}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:43.489216 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:43.496244 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Usage tracker for resource:subnet and tenant:675d4e36e7c14b46864dd445f05bc0ec is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:43.499724 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:43.505860 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:43.529239 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Made reservation on behalf of 675d4e36e7c14b46864dd445f05bc0ec for: {'subnet': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:18:43.531932 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-535570794'] for subnet, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:43.553625 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port ff3663f5-a200-4bed-988a-f47af0d746d7, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=ff3663f5-a200-4bed-988a-f47af0d746d7,segment=NetworkSegment(f3295bfe-6894-409e-b0cc-96311bd90247),segment_id=f3295bfe-6894-409e-b0cc-96311bd90247)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:43.565062 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:18:43.605858 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6d7e949f-e2ab-42d2-9ae8-02991e0fc262 req-21e967a4-84e9-4e2e-a289-4162920819c0 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 210 time: 0.1215179 May 24 11:18:43.689002 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': 'ff3663f5-a200-4bed-988a-f47af0d746d7', 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c', 'port_id': 'ff3663f5-a200-4bed-988a-f47af0d746d7', 'mac_address': 'fa:16:3e:bd:39:db', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 514, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '1ed78d41-8564-48a2-acc1-2ef21df88d9e', 'ip_address': '172.20.0.11'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:18:43.741346 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:43.789452 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:43.797832 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d300dba1-9560-4782-a726-3eeb96033ac5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:43.892506 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d511b52-1121-4a00-a0db-1bee55361553 req-e285e7a9-47fc-4c98-a2cf-8c13c65d77c6 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:43.894112 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6d511b52-1121-4a00-a0db-1bee55361553 req-e285e7a9-47fc-4c98-a2cf-8c13c65d77c6 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 944 time: 0.1964464 May 24 11:18:43.897522 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['45911d6f-9595-40c5-8b29-75259c824bfe', '159fe3b9-426c-472e-9870-2ead1792535a', '516f8f18-b5b5-4ded-bdfd-c420f5651d69'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:18:43.927715 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Network bb3e0f6d-2046-4828-8f43-5f99ba5f3add is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:43.928805 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Pushing event updated for resources: {'Port': ['ID=bb147152-8c96-4e6d-b982-2e6076cc6145,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:43.931958 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:43.934879 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.137s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:43.935736 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:18:43.996295 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Set router 684315b8-ba5e-4d23-8963-64aca3b067cb gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:18:44.007856 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6d511b52-1121-4a00-a0db-1bee55361553 req-6fb872c7-25c6-4b1e-9bdb-7e4779652d8e tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1051445 May 24 11:18:44.034011 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:44.035599 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:44.071386 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.3400507 May 24 11:18:44.086769 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:44.088725 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Request body: {'port': {'device_id': '9233a97a-5cf6-4529-8a81-39073d46adae', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:44.106826 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56194) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:44.133187 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id ff3663f5-a200-4bed-988a-f47af0d746d7 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:18:44.269382 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port ff3663f5-a200-4bed-988a-f47af0d746d7, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=ff3663f5-a200-4bed-988a-f47af0d746d7,segment=NetworkSegment(f3295bfe-6894-409e-b0cc-96311bd90247),segment_id=f3295bfe-6894-409e-b0cc-96311bd90247)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:44.312097 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-534944967', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-535570812'] for subnet, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:44.313417 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:44.320196 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c135e655-3a10-4365-97d0-5b957b88936d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:44.389525 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:44.399117 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:44.403255 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Pushing event updated for resources: {'Subnet': ['ID=a90ff9ef-eadb-4731-b071-d0c9fe13cb00,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:44.407364 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:44.430544 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Checking segment: {'id': '88ac69f8-b455-4003-825b-2df6e05e49fa', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 531, 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:44.431049 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Network hostable DHCP agents. Network: 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91819) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:18:44.445332 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Request body: {'network': {'name': 'tempest-TestVolumeBootPattern-908540324-network', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:44.453422 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:44.467262 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Usage tracker for resource:network and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:44.474637 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:44.479379 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Network 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91819) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:18:44.488021 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:44.501997 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a) {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:44.516344 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:44.516344 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:44.522799 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e35ba9f4-2b80-4c6e-83a8-d6b3f2380df3 None None] Network 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:18:44.525578 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Made reservation on behalf of 9a8d61f79bbc496ca30e390fd9b5839e for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:18:44.528828 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:44.554280 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:44.577979 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 850 time: 1.1063614 May 24 11:18:44.579225 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:44.586280 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56202) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:44.591558 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Usage tracker for resource:security_group_rule and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:44.597161 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Request body: {'router': {'name': 'tempest-TestMinimumBasicScenario-648060998-router', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:44.601459 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:44.608416 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:44.610785 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:44.617304 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Usage tracker for resource:router and tenant:675d4e36e7c14b46864dd445f05bc0ec is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:44.624446 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:44.631604 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:44.661447 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-684314', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4157'] for router, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:44.665287 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-684570', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-687045', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-715560', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4164'] for router, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:44.725192 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=ff3663f5-a200-4bed-988a-f47af0d746d7,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:44.751543 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:44.755395 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.356s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:44.820623 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['ff3663f5-a200-4bed-988a-f47af0d746d7'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:18:44.981558 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id bb147152-8c96-4e6d-b982-2e6076cc6145 {{(pid=91820) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:18:44.991163 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.010387 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ceb529dc-14e7-49de-9d4e-2d9bc6f41615 req-313e58a9-87fd-4f7b-91c0-53c268e978a2 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:45.010680 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-ceb529dc-14e7-49de-9d4e-2d9bc6f41615 req-313e58a9-87fd-4f7b-91c0-53c268e978a2 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Synchronizing usage tracker for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:18:45.034724 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Transition to ACTIVE for port object bb147152-8c96-4e6d-b982-2e6076cc6145 will not be triggered until provisioned by entity L2. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:45.035095 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.041653 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-ceb529dc-14e7-49de-9d4e-2d9bc6f41615 req-313e58a9-87fd-4f7b-91c0-53c268e978a2 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Unset dirty status for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:18:45.043037 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ceb529dc-14e7-49de-9d4e-2d9bc6f41615 req-313e58a9-87fd-4f7b-91c0-53c268e978a2 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 932 time: 0.2640862 May 24 11:18:45.043769 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.054900 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:45.137006 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.154013 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2d9ec83a-f7c9-43d8-b0db-e67728dab0f9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:45.201588 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpea482e27-f15a-5756-a111-7e0dba035002-9672fa3f-743d-4d68-8dd2-0bf684ca5e6a', 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'fixed_ips': [{'subnet_id': 'a90ff9ef-eadb-4731-b071-d0c9fe13cb00'}]}} from dv-cinder-784092-9-iscsi. {{(pid=91821) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 11:18:45.203487 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.205890 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e5fa59cc-2474-401d-98cd-11d70d470d79,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:45.214808 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:45.223984 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:18:39Z', 'revision_number': 4, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '90af9233-b13f-40e5-a0f8-872fe40ddfed', 'name': '', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'mac_address': 'fa:16:3e:ab:4f:f6', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_interface', 'standard_attr_id': 356, 'fixed_ips': [{'subnet_id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:38Z', 'updated_at': '2021-05-24T11:18:40Z', 'revision_number': 2, 'project_id': '788050c389224d17b163f24bb67902bf', 'subnets': [{'id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:45.229058 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] vlan segment allocate from pool success with {'vlan_id': 503, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:18:45.259448 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Checking router: 684315b8-ba5e-4d23-8963-64aca3b067cb for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:18:45.259813 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Updating router 684315b8-ba5e-4d23-8963-64aca3b067cb port 90af9233-b13f-40e5-a0f8-872fe40ddfed binding host dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:18:45.408415 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-687072', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-684328', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4178'] for router, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.430299 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Pushing event updated for resources: {'Port': ['ID=bb147152-8c96-4e6d-b982-2e6076cc6145,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:45.436905 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.382s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:45.447016 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ceb529dc-14e7-49de-9d4e-2d9bc6f41615 req-ce83af03-d0fa-4497-b24d-cbfee775e263 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3546267 May 24 11:18:45.448035 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.473446 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.473978 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Added segment 2f7d3253-3331-43a2-800a-8858592fd924 of type vlan for network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 May 24 11:18:45.509189 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Attempting to bind port bb147152-8c96-4e6d-b982-2e6076cc6145 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:18:45.510311 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Attempting to bind port bb147152-8c96-4e6d-b982-2e6076cc6145 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:18:45.511065 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Attempting to bind port bb147152-8c96-4e6d-b982-2e6076cc6145 on network bb3e0f6d-2046-4828-8f43-5f99ba5f3add {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:45.517582 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Port bb147152-8c96-4e6d-b982-2e6076cc6145 on network bb3e0f6d-2046-4828-8f43-5f99ba5f3add not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:18:45.518458 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Attempting to bind port bb147152-8c96-4e6d-b982-2e6076cc6145 on network bb3e0f6d-2046-4828-8f43-5f99ba5f3add {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:45.526490 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 22), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:18:45.527250 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Checking segment: {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:45.527901 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Bound using segment: {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:18:45.528769 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Bound port: bb147152-8c96-4e6d-b982-2e6076cc6145, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'b756bc07-d062-4b63-a043-c614b00fa2ca', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 524, 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add'}}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:18:45.529373 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.674091 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] For port bb147152-8c96-4e6d-b982-2e6076cc6145, host hv-cinder-78409, cleared binding levels {{(pid=91820) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:18:45.676863 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.678696 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] For port bb147152-8c96-4e6d-b982-2e6076cc6145, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=bb147152-8c96-4e6d-b982-2e6076cc6145,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91820) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:18:45.683764 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.689679 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:45.757681 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:45.758312 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.759937 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] 10.222.0.93 "PUT /v2.0/ports/bb147152-8c96-4e6d-b982-2e6076cc6145 HTTP/1.1" status: 200 len: 1094 time: 1.6794693 May 24 11:18:45.787737 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '9233a97a-5cf6-4529-8a81-39073d46adae', 'tag': 'bb147152-8c96-4e6d-b982-2e6076cc6145'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:18:45.800312 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.db_base_plugin_common [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Allocated IP 172.20.0.2 (9672fa3f-743d-4d68-8dd2-0bf684ca5e6a/a90ff9ef-eadb-4731-b071-d0c9fe13cb00/3b438696-d4a6-404d-a9c9-be9bffe037af) {{(pid=91821) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:18:45.841518 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.859455 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:45.860082 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:45.878217 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:45.892848 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Pushing event updated for resources: {'Port': ['ID=bb147152-8c96-4e6d-b982-2e6076cc6145,revision_number=3']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:45.921074 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.231s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:45.970337 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '9233a97a-5cf6-4529-8a81-39073d46adae', 'tag': 'bb147152-8c96-4e6d-b982-2e6076cc6145', 'status': 'completed', 'code': 200} May 24 11:18:46.010240 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Pushing event updated for resources: {'Network': ['ID=be65d1af-351e-4e5b-8c19-1b04ff5cfc29,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:46.025340 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.147s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:46.034252 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 884 time: 1.9211810 May 24 11:18:46.036491 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 56222) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:46.050515 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Request body: {'subnet': {'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestVolumeBootPattern-908540324-subnet', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'ip_version': 4}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:46.067675 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:18:46.069288 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-11423d1d-f9e7-428e-ba13-680ea90c8bb2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:46.073122 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-11423d1d-f9e7-428e-ba13-680ea90c8bb2 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 1097 time: 0.2512004 May 24 11:18:46.075399 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:46.114768 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-47f06400-3748-41f2-ae53-660de6d8a6c9 req-7c5ffb6b-ed10-487a-993e-26c71346e3da tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:46.125413 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-47f06400-3748-41f2-ae53-660de6d8a6c9 req-7c5ffb6b-ed10-487a-993e-26c71346e3da tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 945 time: 0.7110045 May 24 11:18:46.187428 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Usage tracker for resource:subnet and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:46.232588 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:46.244685 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-6b1d5563-f8a6-44a7-bc6e-823f6c25a9cb service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=bb147152-8c96-4e6d-b982-2e6076cc6145 HTTP/1.1" status: 200 len: 217 time: 0.1628575 May 24 11:18:46.245633 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:46.315745 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Made reservation on behalf of 9a8d61f79bbc496ca30e390fd9b5839e for: {'subnet': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:18:46.318491 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-535456753'] for subnet, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:46.347164 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-47f06400-3748-41f2-ae53-660de6d8a6c9 req-1f267d3c-05ca-4cba-a478-f3daf50444ea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2200842 May 24 11:18:46.428971 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Notify callbacks [] for port, precommit_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:46.431283 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:46.433747 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-a609ed28-fb7f-4723-a53a-85716d638a2c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:46.435203 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-a609ed28-fb7f-4723-a53a-85716d638a2c service neutron] 10.222.0.93 "GET /v2.0/subnets?id=1107b450-6c27-4f13-8b6e-abdbbbb1ea0f HTTP/1.1" status: 200 len: 854 time: 0.1770730 May 24 11:18:46.450410 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Transition to ACTIVE for port object 3b438696-d4a6-404d-a9c9-be9bffe037af will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:46.464542 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:46.511193 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f94ff4cd-4848-4169-bad8-28bd5a52a552 req-0700537a-4383-4f39-92fd-a5a6387f157a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:46.512733 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f94ff4cd-4848-4169-bad8-28bd5a52a552 req-0700537a-4383-4f39-92fd-a5a6387f157a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.3362796 May 24 11:18:46.514926 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-c7db570d-84da-450a-84ce-125420b11ca6 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0708773 May 24 11:18:46.559051 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Transition to ACTIVE for port object 3b438696-d4a6-404d-a9c9-be9bffe037af will not be triggered until provisioned by entity DHCP. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:46.579087 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:46.584787 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Notify callbacks [] for port, precommit_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:46.589579 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:46.669137 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Transition to ACTIVE for port object 90af9233-b13f-40e5-a0f8-872fe40ddfed will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:46.684024 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Network 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a is already hosted by enough agents. {{(pid=91821) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:46.684024 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.base_scheduler [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Resources bound (agent type, host, resource id): {{(pid=91821) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:46.690409 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Received port after_create {{(pid=91821) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:18:46.704726 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:46.719957 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Pushing event updated for resources: {'Port': ['ID=3b438696-d4a6-404d-a9c9-be9bffe037af,revision_number=1']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:46.726938 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f94ff4cd-4848-4169-bad8-28bd5a52a552 req-0ed319d4-be88-4df6-b495-f36cf32d07e9 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1974692 May 24 11:18:46.770863 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.181s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:46.793827 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Allocated IP 172.27.0.181 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/23bbb434-e051-4fbc-8e80-f213cd1ab793) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:18:46.835253 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88ef8888-4945-45c5-b083-2ac1643685df req-291c2fba-aa0b-494b-abbb-d28d49a26ff8 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:46.838071 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-f2843fca-634d-4d56-b25c-38fae8969882 service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3052530 May 24 11:18:46.848498 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-88ef8888-4945-45c5-b083-2ac1643685df req-291c2fba-aa0b-494b-abbb-d28d49a26ff8 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 944 time: 0.2282314 May 24 11:18:46.875291 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.151s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:46.977205 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c285c667-1f23-4119-b92b-c1cf20d046f7 None None] Pushing event updated for resources: {'Port': ['ID=90af9233-b13f-40e5-a0f8-872fe40ddfed,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:46.978247 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-88ef8888-4945-45c5-b083-2ac1643685df req-c01b7f52-a5ad-4e6b-8f32-2bcf9137ec91 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1281507 May 24 11:18:46.986136 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device ff3663f5-a200-4bed-988a-f47af0d746d7 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:47.079812 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c285c667-1f23-4119-b92b-c1cf20d046f7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.204s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:47.096933 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8b3c5256-adae-40ea-ab4e-9c949a4fdfbf req-63763ab4-6c9d-4cee-8e22-a701c3ac08dd service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2507830 May 24 11:18:47.173422 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:47.200156 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Attempting to bind port 3b438696-d4a6-404d-a9c9-be9bffe037af on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:18:47.200530 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Attempting to bind port 3b438696-d4a6-404d-a9c9-be9bffe037af by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '88ac69f8-b455-4003-825b-2df6e05e49fa', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 531, 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:18:47.200891 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Attempting to bind port 3b438696-d4a6-404d-a9c9-be9bffe037af on network 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:47.211362 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:47.218291 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:47.223611 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:18:47.223992 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Checking segment: {'id': '88ac69f8-b455-4003-825b-2df6e05e49fa', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 531, 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:47.224395 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:47.224786 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Bound using segment: {'id': '88ac69f8-b455-4003-825b-2df6e05e49fa', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 531, 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:18:47.225291 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Bound port: 3b438696-d4a6-404d-a9c9-be9bffe037af, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '88ac69f8-b455-4003-825b-2df6e05e49fa', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 531, 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:18:47.225668 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:47.244999 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Attempting to bind port 90af9233-b13f-40e5-a0f8-872fe40ddfed on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:18:47.245396 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Attempting to bind port 90af9233-b13f-40e5-a0f8-872fe40ddfed by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:18:47.245763 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Attempting to bind port 90af9233-b13f-40e5-a0f8-872fe40ddfed on network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:47.287429 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:18:47.287807 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Checking segment: {'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:47.288222 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:47.288628 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Bound using segment: {'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:18:47.289133 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Bound port: 90af9233-b13f-40e5-a0f8-872fe40ddfed, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:18:47.289519 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:47.293194 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-534788555', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-535456743'] for subnet, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:47.298715 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:18:47.301375 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1446dcc6-3038-49c6-8a2f-a34364600b9b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:47.305947 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port ff3663f5-a200-4bed-988a-f47af0d746d7 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:47.306934 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:47.310503 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:47.313910 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:18:47.327841 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port ff3663f5-a200-4bed-988a-f47af0d746d7 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:47.328216 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:47.348059 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6a1e2af6-4453-4925-aa9a-617c527e2da8 req-06dfb6f2-4b2f-4139-8193-f28fbfe842c4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:47.349529 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6a1e2af6-4453-4925-aa9a-617c527e2da8 req-06dfb6f2-4b2f-4139-8193-f28fbfe842c4 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 1097 time: 0.1872809 May 24 11:18:47.363822 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Pushing event updated for resources: {'Port': ['ID=23bbb434-e051-4fbc-8e80-f213cd1ab793,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:47.380682 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.162s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:47.462262 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] For port 3b438696-d4a6-404d-a9c9-be9bffe037af, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:18:47.469384 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] For port 3b438696-d4a6-404d-a9c9-be9bffe037af, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=3b438696-d4a6-404d-a9c9-be9bffe037af,segment=NetworkSegment(88ac69f8-b455-4003-825b-2df6e05e49fa),segment_id=88ac69f8-b455-4003-825b-2df6e05e49fa)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:18:47.505742 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:47.510750 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] For port 90af9233-b13f-40e5-a0f8-872fe40ddfed, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:18:47.513128 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:47.622410 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] For port 90af9233-b13f-40e5-a0f8-872fe40ddfed, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=90af9233-b13f-40e5-a0f8-872fe40ddfed,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:18:47.638032 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Pushing event updated for resources: {'Subnet': ['ID=994391be-baae-4f60-9ef9-bf02d19d3477,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:47.645369 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:47.649179 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.348s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:47.656349 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Checking segment: {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:47.656702 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Network hostable DHCP agents. Network: be65d1af-351e-4e5b-8c19-1b04ff5cfc29, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91820) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:18:47.707313 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Pushing event updated for resources: {'Port': ['ID=3b438696-d4a6-404d-a9c9-be9bffe037af,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:47.722468 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91820) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:18:47.722678 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.209s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:47.725244 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.074s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:47.736800 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6a1e2af6-4453-4925-aa9a-617c527e2da8 req-10544d6b-723f-4672-85e9-811835ef936d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:47.738445 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6a1e2af6-4453-4925-aa9a-617c527e2da8 req-10544d6b-723f-4672-85e9-811835ef936d service neutron] 10.222.0.93 "GET /v2.0/networks?id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add HTTP/1.1" status: 200 len: 915 time: 0.3805125 May 24 11:18:47.747345 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, be65d1af-351e-4e5b-8c19-1b04ff5cfc29) {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:47.798239 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-ddd24dd5-41bc-435e-ae2c-7010edaaa6f5 None None] Network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:18:47.818926 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:47.847399 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6a1e2af6-4453-4925-aa9a-617c527e2da8 req-e31677bc-e369-4e92-b1b4-45bc60367743 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=bb147152-8c96-4e6d-b982-2e6076cc6145 HTTP/1.1" status: 200 len: 217 time: 0.0918331 May 24 11:18:47.857686 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d6067e4d-3ee2-432f-a776-2099362caa8d req-4d9b6a8d-2d90-4ac1-aa97-4f9746b5cf6b tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:47.862292 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d6067e4d-3ee2-432f-a776-2099362caa8d req-4d9b6a8d-2d90-4ac1-aa97-4f9746b5cf6b tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 945 time: 0.3400993 May 24 11:18:47.863820 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:47.864603 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:47.968197 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-0bb00d87-2a18-4361-82c1-0c431d452942 admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 847 time: 1.9290724 May 24 11:18:47.976593 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56234) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:47.990292 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d6067e4d-3ee2-432f-a776-2099362caa8d req-cf86772c-e325-4670-8a6a-b02c41dd2fa0 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1179836 May 24 11:18:47.994330 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6a1e2af6-4453-4925-aa9a-617c527e2da8 req-0c8a5e8f-7121-445e-995d-9a2aa771d279 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:47.996127 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6a1e2af6-4453-4925-aa9a-617c527e2da8 req-0c8a5e8f-7121-445e-995d-9a2aa771d279 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=1107b450-6c27-4f13-8b6e-abdbbbb1ea0f HTTP/1.1" status: 200 len: 854 time: 0.1269398 May 24 11:18:48.006911 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Request body: {'router': {'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:18:48.033311 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:18:48.073373 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Usage tracker for resource:router and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:18:48.080562 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6a1e2af6-4453-4925-aa9a-617c527e2da8 req-8bd6baaf-9859-4c7b-bd9d-343486c434ff service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=bb3e0f6d-2046-4828-8f43-5f99ba5f3add&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0755537 May 24 11:18:48.084601 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:18:48.127724 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:18:48.159546 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:48.168450 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Routers {'684315b8-ba5e-4d23-8963-64aca3b067cb'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:18:48.169132 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['684315b8-ba5e-4d23-8963-64aca3b067cb'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:48.245170 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c285c667-1f23-4119-b92b-c1cf20d046f7 None None] Pushing event updated for resources: {'Port': ['ID=90af9233-b13f-40e5-a0f8-872fe40ddfed,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:48.249072 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c285c667-1f23-4119-b92b-c1cf20d046f7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.523s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:48.271435 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-684314', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4157'] for router, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:48.273367 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-684570', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-687045', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-715560', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4164'] for router, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:48.354462 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '78ccab16-0ff0-4b84-8d91-71e4092110e9', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'ff3663f5-a200-4bed-988a-f47af0d746d7'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:18:48.360365 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpea482e27-f15a-5756-a111-7e0dba035002-be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477'}]}} from dv-cinder-784092-9-iscsi. {{(pid=91821) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 11:18:48.361336 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:48.369217 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6a1e2af6-4453-4925-aa9a-617c527e2da8 req-12660a84-5e04-47cb-869f-7c89ab617c46 service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2794371 May 24 11:18:48.389538 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port ff3663f5-a200-4bed-988a-f47af0d746d7, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=ff3663f5-a200-4bed-988a-f47af0d746d7,segment=NetworkSegment(f3295bfe-6894-409e-b0cc-96311bd90247),segment_id=f3295bfe-6894-409e-b0cc-96311bd90247)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:48.408989 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-992d22e2-83cb-47e1-a6da-4f7c8dbc83c7 req-a97c56f9-ce4e-4a2e-bd73-f8737e09e4c1 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:48.417028 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-992d22e2-83cb-47e1-a6da-4f7c8dbc83c7 req-a97c56f9-ce4e-4a2e-bd73-f8737e09e4c1 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 944 time: 0.1801889 May 24 11:18:48.446850 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:48.451216 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:48.462471 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:48.554166 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-992d22e2-83cb-47e1-a6da-4f7c8dbc83c7 req-21a38933-8ad4-4991-bcf1-0bbcc8a57677 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1361494 May 24 11:18:48.607300 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6a1e2af6-4453-4925-aa9a-617c527e2da8 req-534b74f8-1028-48a3-b104-1a997a252703 service neutron] 10.222.0.93 "GET /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2274776 May 24 11:18:48.609345 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '78ccab16-0ff0-4b84-8d91-71e4092110e9', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'ff3663f5-a200-4bed-988a-f47af0d746d7', 'code': 200} May 24 11:18:48.688097 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=ff3663f5-a200-4bed-988a-f47af0d746d7,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:48.694905 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29cbd56e-0d19-4829-939f-e2d38c208f71 req-040ff940-75d0-43ec-8568-3b87be6f367a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:48.696464 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-29cbd56e-0d19-4829-939f-e2d38c208f71 req-040ff940-75d0-43ec-8568-3b87be6f367a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.7664423 May 24 11:18:48.705713 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.230s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:48.740619 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] Set router 684315b8-ba5e-4d23-8963-64aca3b067cb gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:18:48.810354 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.db_base_plugin_common [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Allocated IP 172.20.0.2 (be65d1af-351e-4e5b-8c19-1b04ff5cfc29/994391be-baae-4f60-9ef9-bf02d19d3477/4e43bacd-47d7-4c30-b3b3-64bc8458a6dc) {{(pid=91821) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:18:48.924009 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device bb147152-8c96-4e6d-b982-2e6076cc6145 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:18:48.974748 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-29cbd56e-0d19-4829-939f-e2d38c208f71 req-0d2ca7aa-102c-4e22-ab76-aff24cee50aa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2140725 May 24 11:18:49.002398 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-684307', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-687011'] for router, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:49.030533 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-80028f83-9e68-456b-a741-06b239bf1fd2 req-6ab15042-56a8-4286-adf0-3a58ad5ec0a6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:49.032203 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-80028f83-9e68-456b-a741-06b239bf1fd2 req-6ab15042-56a8-4286-adf0-3a58ad5ec0a6 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 1099 time: 0.1877782 May 24 11:18:49.085169 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 3b438696-d4a6-404d-a9c9-be9bffe037af up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:49.128135 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-80028f83-9e68-456b-a741-06b239bf1fd2 req-dddbbed5-1a74-4d66-b2bf-8e4b45a4545f service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0839634 May 24 11:18:49.178074 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-4045cd15-b52f-4743-b9b4-7f59166e5fcb None None] Network 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:18:49.259760 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-51a95556-ac3b-43b2-b14f-8378402536bc req-90d66196-9fd6-4dd0-8798-6e3dcd96f84a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:49.261556 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-51a95556-ac3b-43b2-b14f-8378402536bc req-90d66196-9fd6-4dd0-8798-6e3dcd96f84a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 946 time: 0.1133876 May 24 11:18:49.272031 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-687072', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-684328', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4178'] for router, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:49.321198 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-51a95556-ac3b-43b2-b14f-8378402536bc req-9301fc50-0ea1-4089-add5-b78a90e3d41a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0513277 May 24 11:18:49.456736 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Router d37dc0a7-5e53-4516-8020-b2990086e8c5 is scheduled to L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 with binding_index 1 {{(pid=91819) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 11:18:49.556989 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-e6b8d1af-e4ba-402f-a0e3-26d3700ce2e7 None None] Provisioning for port 3b438696-d4a6-404d-a9c9-be9bffe037af completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:49.765436 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 3b438696-d4a6-404d-a9c9-be9bffe037af completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:49.792476 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port bb147152-8c96-4e6d-b982-2e6076cc6145, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=bb147152-8c96-4e6d-b982-2e6076cc6145,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:49.801065 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 3b438696-d4a6-404d-a9c9-be9bffe037af triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:49.801399 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:49.834657 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:18:49.845573 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Sync routers for ids ['d37dc0a7-5e53-4516-8020-b2990086e8c5'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:18:49.856968 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Routers {'d37dc0a7-5e53-4516-8020-b2990086e8c5'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:18:49.857616 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['d37dc0a7-5e53-4516-8020-b2990086e8c5'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:49.880449 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': 'bb147152-8c96-4e6d-b982-2e6076cc6145', 'network_id': 'bb3e0f6d-2046-4828-8f43-5f99ba5f3add', 'port_id': 'bb147152-8c96-4e6d-b982-2e6076cc6145', 'mac_address': 'fa:16:3e:b5:09:d3', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 524, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '1107b450-6c27-4f13-8b6e-abdbbbb1ea0f', 'ip_address': '172.20.0.10'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:18:49.888174 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Notify callbacks [] for port, precommit_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:49.920857 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Notify callbacks [] for router, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:49.925008 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] 10.222.0.18 "POST /v2.0/routers HTTP/1.1" status: 201 len: 926 time: 5.3325839 May 24 11:18:49.926092 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 56250) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:49.939190 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:49.977618 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:49.984989 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Transition to ACTIVE for port object 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:49.988548 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e1b74b9-58b4-4c87-a6ae-c9fd73dda593 req-222bb495-3a0b-48dc-9c8b-aa913d47baa3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:49.993712 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0e1b74b9-58b4-4c87-a6ae-c9fd73dda593 req-222bb495-3a0b-48dc-9c8b-aa913d47baa3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 944 time: 0.2769151 May 24 11:18:50.059129 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Transition to ACTIVE for port object 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc will not be triggered until provisioned by entity DHCP. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:50.083386 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:50.098292 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:50.107354 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:50.150299 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:18:39Z', 'revision_number': 4, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '90af9233-b13f-40e5-a0f8-872fe40ddfed', 'name': '', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'mac_address': 'fa:16:3e:ab:4f:f6', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_interface', 'standard_attr_id': 356, 'fixed_ips': [{'subnet_id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:38Z', 'updated_at': '2021-05-24T11:18:47Z', 'revision_number': 4, 'project_id': '788050c389224d17b163f24bb67902bf', 'subnets': [{'id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:50.255308 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id bb147152-8c96-4e6d-b982-2e6076cc6145 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:18:50.259905 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f90fcaa7-c656-45af-85ec-628587cbad2b None None] The sync data for ids ['684315b8-ba5e-4d23-8963-64aca3b067cb'] in dv-cinder-784092-9-iscsi is: [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:18:39Z', 'revision_number': 4, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '90af9233-b13f-40e5-a0f8-872fe40ddfed', 'name': '', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'mac_address': 'fa:16:3e:ab:4f:f6', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_interface', 'standard_attr_id': 356, 'fixed_ips': [{'subnet_id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:38Z', 'updated_at': '2021-05-24T11:18:47Z', 'revision_number': 4, 'project_id': '788050c389224d17b163f24bb67902bf', 'subnets': [{'id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:18:50.280036 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0e1b74b9-58b4-4c87-a6ae-c9fd73dda593 req-7040ae36-a796-48e0-861d-23d3a3351c53 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1672082 May 24 11:18:50.281998 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Set router d37dc0a7-5e53-4516-8020-b2990086e8c5 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:18:50.359636 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 is already hosted by enough agents. {{(pid=91821) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:50.368591 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Allocated IP 172.27.0.29 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/e648ddfe-6f08-4b5c-8c21-e6794aa56080) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:18:50.371519 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.base_scheduler [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Resources bound (agent type, host, resource id): {{(pid=91821) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:50.382265 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port bb147152-8c96-4e6d-b982-2e6076cc6145, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=bb147152-8c96-4e6d-b982-2e6076cc6145,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:50.384377 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Received port after_create {{(pid=91821) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:18:50.386746 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:50.399670 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Pushing event updated for resources: {'Port': ['ID=4e43bacd-47d7-4c30-b3b3-64bc8458a6dc,revision_number=1']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:50.407404 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.300s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:50.412549 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:50.425909 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c285c667-1f23-4119-b92b-c1cf20d046f7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:50.791907 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:50.814295 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=bb147152-8c96-4e6d-b982-2e6076cc6145,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:50.827560 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.402s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:50.830067 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0206aa5-503c-40b7-a8b5-89f98b7a7463 req-44774195-b051-4a4d-a0eb-9b8f7b6c6084 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:50.831713 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e0206aa5-503c-40b7-a8b5-89f98b7a7463 req-44774195-b051-4a4d-a0eb-9b8f7b6c6084 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 946 time: 0.2189765 May 24 11:18:50.842688 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 3b438696-d4a6-404d-a9c9-be9bffe037af, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=3b438696-d4a6-404d-a9c9-be9bffe037af,segment=NetworkSegment(88ac69f8-b455-4003-825b-2df6e05e49fa),segment_id=88ac69f8-b455-4003-825b-2df6e05e49fa)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:50.855373 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['bb147152-8c96-4e6d-b982-2e6076cc6145'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:18:50.918612 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:50.926146 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Attempting to bind port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:18:50.926504 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Attempting to bind port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:18:50.926880 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Attempting to bind port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc on network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:50.935197 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:50.963599 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:50.982041 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:51.022661 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:18:51.023039 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Checking segment: {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:51.023454 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:51.023907 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Bound using segment: {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:18:51.024404 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Bound port: 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:18:51.024785 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:51.061874 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e0206aa5-503c-40b7-a8b5-89f98b7a7463 req-bb7c8dde-66ae-45f1-9019-e21925741fe0 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2003660 May 24 11:18:51.061874 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:51.066620 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:51.111264 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:18:51.111927 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'name': 'tempest-TestMinimumBasicScenario-648060998-router', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181'}], 'enable_snat': True}, 'gw_port_id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:44Z', 'updated_at': '2021-05-24T11:18:47Z', 'revision_number': 3, 'project_id': '675d4e36e7c14b46864dd445f05bc0ec', 'gw_port': {'id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:04:4a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 385, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:46Z', 'updated_at': '2021-05-24T11:18:46Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:51.122296 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Checking router: d37dc0a7-5e53-4516-8020-b2990086e8c5 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:18:51.122745 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Updating router d37dc0a7-5e53-4516-8020-b2990086e8c5 port 23bbb434-e051-4fbc-8e80-f213cd1ab793 binding host dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:18:51.183558 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 90af9233-b13f-40e5-a0f8-872fe40ddfed up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:51.251955 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:51.263621 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-6966f819-1b0a-4373-8af7-2410608561a0 None None] Provisioning complete for port 3b438696-d4a6-404d-a9c9-be9bffe037af triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:51.264004 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-6966f819-1b0a-4373-8af7-2410608561a0 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:51.265342 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Pushing event updated for resources: {'Port': ['ID=e648ddfe-6f08-4b5c-8c21-e6794aa56080,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:51.270643 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.336s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:51.377652 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=3b438696-d4a6-404d-a9c9-be9bffe037af,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:51.418055 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.436s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:51.492327 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] For port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:18:51.498659 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] For port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=4e43bacd-47d7-4c30-b3b3-64bc8458a6dc,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:18:51.526212 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:51.540458 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:51.672339 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ba8a881d-2898-42f5-b2d4-5163ea7ff63f req-ecd32ad4-680e-47bd-b646-3dd588731c15 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:51.673958 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ba8a881d-2898-42f5-b2d4-5163ea7ff63f req-ecd32ad4-680e-47bd-b646-3dd588731c15 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 1.1179330 May 24 11:18:51.702667 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 90af9233-b13f-40e5-a0f8-872fe40ddfed completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:51.787819 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:51.811486 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 90af9233-b13f-40e5-a0f8-872fe40ddfed triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:51.814100 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:51.932858 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Notify callbacks [] for port, precommit_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:51.970763 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.provisioning_blocks [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Transition to ACTIVE for port object 23bbb434-e051-4fbc-8e80-f213cd1ab793 will not be triggered until provisioned by entity L2. {{(pid=91822) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:51.986965 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:51.998699 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-ab82f997-fd89-44b5-89b1-1a8f73cd839c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.003s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:52.041420 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Pushing event updated for resources: {'Port': ['ID=4e43bacd-47d7-4c30-b3b3-64bc8458a6dc,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:52.049373 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.509s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:52.089710 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:52.190345 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ba8a881d-2898-42f5-b2d4-5163ea7ff63f req-8097226a-f95f-4968-838e-f418990dadc2 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.4931502 May 24 11:18:52.220612 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:52.267733 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Attempting to bind port 23bbb434-e051-4fbc-8e80-f213cd1ab793 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:18:52.268236 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Attempting to bind port 23bbb434-e051-4fbc-8e80-f213cd1ab793 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:18:52.268464 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Attempting to bind port 23bbb434-e051-4fbc-8e80-f213cd1ab793 on network 858b8fad-a73f-4578-9079-6e731b9ff7e2 {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:52.308833 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-52070a7e-091b-40cb-b656-050f232a2091 None None] Pushing event updated for resources: {'Port': ['ID=23bbb434-e051-4fbc-8e80-f213cd1ab793,revision_number=2']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:52.319286 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-52070a7e-091b-40cb-b656-050f232a2091 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.322s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:52.328206 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:18:52.329334 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Checking segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91822) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:52.329950 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:52.330278 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Bound using segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:18:52.330688 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Bound port: 23bbb434-e051-4fbc-8e80-f213cd1ab793, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:18:52.331223 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:52.368884 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 90af9233-b13f-40e5-a0f8-872fe40ddfed, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=90af9233-b13f-40e5-a0f8-872fe40ddfed,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:52.377249 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:52.381984 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49c52d33-20b5-43e0-a89f-dcc6ae1cda84 req-1a562e99-b8d0-4dcb-9c41-bc305e5e1378 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:52.385079 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-49c52d33-20b5-43e0-a89f-dcc6ae1cda84 req-1a562e99-b8d0-4dcb-9c41-bc305e5e1378 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 945 time: 0.6040897 May 24 11:18:52.386868 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c7541d61-630d-44aa-8da8-12fb967b6e0b None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:52.458335 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Allocated IP 172.20.0.1 (9672fa3f-743d-4d68-8dd2-0bf684ca5e6a/a90ff9ef-eadb-4731-b071-d0c9fe13cb00/1a4dddfd-3c51-421f-b942-92a179e987b8) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:18:52.491552 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:52.543404 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=90af9233-b13f-40e5-a0f8-872fe40ddfed,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:52.549612 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.170s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:52.564600 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] For port 23bbb434-e051-4fbc-8e80-f213cd1ab793, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91822) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:18:52.578784 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] For port 23bbb434-e051-4fbc-8e80-f213cd1ab793, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=23bbb434-e051-4fbc-8e80-f213cd1ab793,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91822) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:18:52.584563 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:52.594807 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-52070a7e-091b-40cb-b656-050f232a2091 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:52.624117 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-49c52d33-20b5-43e0-a89f-dcc6ae1cda84 req-bf1cbc0c-ee21-401c-8643-5114e79fd124 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2323997 May 24 11:18:52.675403 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Routers {'d37dc0a7-5e53-4516-8020-b2990086e8c5'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:18:52.676085 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['d37dc0a7-5e53-4516-8020-b2990086e8c5'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:52.730166 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:52.752857 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Transition to ACTIVE for port object 1a4dddfd-3c51-421f-b942-92a179e987b8 will not be triggered until provisioned by entity DHCP. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:52.788555 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:52.795238 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e1c5eda2-bca9-40e3-8bf5-a39a948e9eff admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:52.820589 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-52070a7e-091b-40cb-b656-050f232a2091 None None] Pushing event updated for resources: {'Port': ['ID=23bbb434-e051-4fbc-8e80-f213cd1ab793,revision_number=3']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:52.835463 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-52070a7e-091b-40cb-b656-050f232a2091 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.241s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:52.892134 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Network 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:52.894985 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] Set router d37dc0a7-5e53-4516-8020-b2990086e8c5 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:18:52.897620 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:52.904344 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:18:53.015218 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Pushing event updated for resources: {'Port': ['ID=1a4dddfd-3c51-421f-b942-92a179e987b8,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:53.022298 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.229s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:53.028212 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9965aca7-651b-44f0-aaa7-af03ce7757b2 req-c0febce0-6691-4d9d-bcf8-3b66e1fa3a19 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:53.029796 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9965aca7-651b-44f0-aaa7-af03ce7757b2 req-c0febce0-6691-4d9d-bcf8-3b66e1fa3a19 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 946 time: 0.1588142 May 24 11:18:53.030849 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-684307', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-687011'] for router, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:53.076617 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-687570'] for router_interface, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:53.270865 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 3b438696-d4a6-404d-a9c9-be9bffe037af up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:53.337891 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:53.385049 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-cdc188fe-72e2-46c7-b876-f08bd1bc9db2 None None] Provisioning for port 1a4dddfd-3c51-421f-b942-92a179e987b8 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:53.407885 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-cdc188fe-72e2-46c7-b876-f08bd1bc9db2 None None] Provisioning complete for port 1a4dddfd-3c51-421f-b942-92a179e987b8 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:53.408320 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-cdc188fe-72e2-46c7-b876-f08bd1bc9db2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:53.409723 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'name': 'tempest-TestMinimumBasicScenario-648060998-router', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181'}], 'enable_snat': True}, 'gw_port_id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:44Z', 'updated_at': '2021-05-24T11:18:47Z', 'revision_number': 3, 'project_id': '675d4e36e7c14b46864dd445f05bc0ec', 'gw_port': {'id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:04:4a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 385, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:46Z', 'updated_at': '2021-05-24T11:18:52Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:53.416256 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-55f3430f-caeb-456a-b2d8-8bc7e656d835 None None] The sync data for ids ['d37dc0a7-5e53-4516-8020-b2990086e8c5'] in dv-cinder-784092-9-iscsi is: [{'id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'name': 'tempest-TestMinimumBasicScenario-648060998-router', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181'}], 'enable_snat': True}, 'gw_port_id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:44Z', 'updated_at': '2021-05-24T11:18:47Z', 'revision_number': 3, 'project_id': '675d4e36e7c14b46864dd445f05bc0ec', 'gw_port': {'id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:04:4a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 385, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:46Z', 'updated_at': '2021-05-24T11:18:52Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:18:53.438359 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9965aca7-651b-44f0-aaa7-af03ce7757b2 req-4454159f-cfc9-49fa-b384-31306ff0134a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3208861 May 24 11:18:53.547751 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Router 81705dbf-841d-45b4-8b2d-afa9886a8e59 is scheduled to L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 with binding_index 1 {{(pid=91819) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 11:18:53.615591 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 3b438696-d4a6-404d-a9c9-be9bffe037af triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:53.616051 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:53.656525 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-6fd64c0f-c5e2-4cc5-8a3a-73b1f96403e4 None None] Network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:18:53.703170 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-cdc188fe-72e2-46c7-b876-f08bd1bc9db2 None None] Port 1a4dddfd-3c51-421f-b942-92a179e987b8 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:18:53.707567 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e403c129-504e-41ac-8710-eaf056f49197 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:53.825500 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-39306a89-1bdc-486f-8074-a1316c756617 req-63f670cc-acd2-4bdd-9ac2-e10dab51d64e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:53.827390 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-39306a89-1bdc-486f-8074-a1316c756617 req-63f670cc-acd2-4bdd-9ac2-e10dab51d64e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.3708553 May 24 11:18:53.856806 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:18:53.919731 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Notify callbacks [] for router, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:53.924853 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] 10.222.0.18 "POST /v2.0/routers HTTP/1.1" status: 201 len: 922 time: 5.9388981 May 24 11:18:53.925216 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56286) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:18:53.986141 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-39306a89-1bdc-486f-8074-a1316c756617 req-b86348a3-dbf6-4f7b-95c1-a83cb8a65351 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1105886 May 24 11:18:54.030944 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:54.055807 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:54.108843 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Sync routers for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:18:54.139408 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:54.203002 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:18:54.205638 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:54.376630 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Pushing event updated for resources: {'Port': ['ID=1a4dddfd-3c51-421f-b942-92a179e987b8,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:54.380007 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.241s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:54.383537 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-533332', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-687487'] for router_interface, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:54.389286 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-19de224c-dd2c-47fe-9c80-7ce2d13ce1e2 None None] Provisioning for port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:54.524278 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3e7873c8-5f18-4924-804c-7628780fc5b1 req-23406afe-0d46-44c3-8794-ec4ea6bfb6b0 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:54.532550 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3e7873c8-5f18-4924-804c-7628780fc5b1 req-23406afe-0d46-44c3-8794-ec4ea6bfb6b0 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.2551763 May 24 11:18:54.578854 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:18:54.608163 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7c55481d-fc64-4ff0-9592-ab026854a0bb req-30376ae3-fea0-4fdb-aaa7-c390a1320c6b tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:54.609903 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7c55481d-fc64-4ff0-9592-ab026854a0bb req-30376ae3-fea0-4fdb-aaa7-c390a1320c6b tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 945 time: 0.3735685 May 24 11:18:54.650023 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8ceb25d3-0554-4d9e-85a8-948b5e378954 req-dcf654ff-3e2c-4b7d-86dc-3fe376b3bc30 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:54.651497 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8ceb25d3-0554-4d9e-85a8-948b5e378954 req-dcf654ff-3e2c-4b7d-86dc-3fe376b3bc30 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 1099 time: 0.4709482 May 24 11:18:54.707979 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 90af9233-b13f-40e5-a0f8-872fe40ddfed up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:54.730917 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7bca9cf0-fe51-4c9a-b63a-722c6dbd50c2 req-a60fbaf4-b256-4354-a436-5441228a6c63 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:54.732944 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7bca9cf0-fe51-4c9a-b63a-722c6dbd50c2 req-a60fbaf4-b256-4354-a436-5441228a6c63 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.93 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 946 time: 0.4569438 May 24 11:18:54.775555 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7c55481d-fc64-4ff0-9592-ab026854a0bb req-49e19ec3-f2a6-4a81-b5f0-1e2962f4d36e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1501374 May 24 11:18:54.893728 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Network 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:54.900096 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:54.921296 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 90af9233-b13f-40e5-a0f8-872fe40ddfed triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:54.921878 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:54.955826 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device bb147152-8c96-4e6d-b982-2e6076cc6145 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:55.064799 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8ceb25d3-0554-4d9e-85a8-948b5e378954 req-0bbb4af2-7551-43a7-bdec-7da03c0b80e6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:55.067323 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8ceb25d3-0554-4d9e-85a8-948b5e378954 req-0bbb4af2-7551-43a7-bdec-7da03c0b80e6 service neutron] 10.222.0.93 "GET /v2.0/networks?id=4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 200 len: 918 time: 0.4079459 May 24 11:18:55.069201 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3e7873c8-5f18-4924-804c-7628780fc5b1 req-daaa0380-d27d-4810-970e-8c71c36ae6fe tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.5353298 May 24 11:18:55.178370 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-7bca9cf0-fe51-4c9a-b63a-722c6dbd50c2 req-9c3549f5-3b13-4f88-89e3-cd954f0a824a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Deleting port ff3663f5-a200-4bed-988a-f47af0d746d7 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:18:55.178740 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-7bca9cf0-fe51-4c9a-b63a-722c6dbd50c2 req-9c3549f5-3b13-4f88-89e3-cd954f0a824a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:55.213569 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port bb147152-8c96-4e6d-b982-2e6076cc6145 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:55.225585 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port bb147152-8c96-4e6d-b982-2e6076cc6145 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:55.226050 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:55.340912 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8ceb25d3-0554-4d9e-85a8-948b5e378954 req-88df0ecf-e244-4c2c-bc45-2ad69b57d1a5 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=02210ade-8c60-41b6-aa82-a104d4fc0447 HTTP/1.1" status: 200 len: 217 time: 0.0976007 May 24 11:18:55.390215 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:51Z', 'revision_number': 3, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:18:50Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:55.440238 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:55.448433 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-525d2be1-a808-4ef9-a545-650d9165b3e4 req-85b8b057-0397-4b76-8004-5557a40e5ae1 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:55.455898 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-525d2be1-a808-4ef9-a545-650d9165b3e4 req-85b8b057-0397-4b76-8004-5557a40e5ae1 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 946 time: 0.5438178 May 24 11:18:55.466816 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Checking router: 81705dbf-841d-45b4-8b2d-afa9886a8e59 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:18:55.466816 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Updating router 81705dbf-841d-45b4-8b2d-afa9886a8e59 port e648ddfe-6f08-4b5c-8c21-e6794aa56080 binding host dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:18:55.483169 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8ceb25d3-0554-4d9e-85a8-948b5e378954 req-db30bb33-f6c4-4a92-9bfe-6c8da0fa0f7d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:55.484675 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8ceb25d3-0554-4d9e-85a8-948b5e378954 req-db30bb33-f6c4-4a92-9bfe-6c8da0fa0f7d service neutron] 10.222.0.93 "GET /v2.0/subnets?id=52267672-fd8c-45bd-89ea-d9d02a7b249a HTTP/1.1" status: 200 len: 857 time: 0.1329875 May 24 11:18:55.561649 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Router d37dc0a7-5e53-4516-8020-b2990086e8c5 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:18:55.576641 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8ceb25d3-0554-4d9e-85a8-948b5e378954 req-7b670e7e-f9d0-4d39-baf3-eba07713ab09 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=4fec6b68-64f1-4553-8ca5-f86c63ac9714&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0779083 May 24 11:18:55.589816 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:55.593387 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-525d2be1-a808-4ef9-a545-650d9165b3e4 req-4805db49-7d3f-4be8-aa3a-e1856cd367bc tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1364524 May 24 11:18:55.627979 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:55.634124 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-7bca9cf0-fe51-4c9a-b63a-722c6dbd50c2 req-9c3549f5-3b13-4f88-89e3-cd954f0a824a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] For port ff3663f5-a200-4bed-988a-f47af0d746d7, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=ff3663f5-a200-4bed-988a-f47af0d746d7,segment=NetworkSegment(f3295bfe-6894-409e-b0cc-96311bd90247),segment_id=f3295bfe-6894-409e-b0cc-96311bd90247)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:55.634719 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-7bca9cf0-fe51-4c9a-b63a-722c6dbd50c2 req-9c3549f5-3b13-4f88-89e3-cd954f0a824a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:55.667706 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca673945-bf9d-438a-9eb7-c9a9ced1da47 req-80660940-7ca2-4bd6-b75e-2f376b89bb96 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:55.669575 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ca673945-bf9d-438a-9eb7-c9a9ced1da47 req-80660940-7ca2-4bd6-b75e-2f376b89bb96 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.3299947 May 24 11:18:55.703646 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-ccbb0f25-23fa-4025-abec-df5d4ea591d3 None None] Provisioning complete for port 1a4dddfd-3c51-421f-b942-92a179e987b8 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:55.704014 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-ccbb0f25-23fa-4025-abec-df5d4ea591d3 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:55.795831 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9233a97a-5cf6-4529-8a81-39073d46adae', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'bb147152-8c96-4e6d-b982-2e6076cc6145'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:18:55.807371 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-7bca9cf0-fe51-4c9a-b63a-722c6dbd50c2 req-9c3549f5-3b13-4f88-89e3-cd954f0a824a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Calling delete_port for ff3663f5-a200-4bed-988a-f47af0d746d7 owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:18:55.815536 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:55.905545 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:55.905930 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:55.923955 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8ceb25d3-0554-4d9e-85a8-948b5e378954 req-382382a5-8766-4b9e-b629-268f852e2fd5 service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3296041 May 24 11:18:55.932984 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port bb147152-8c96-4e6d-b982-2e6076cc6145, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=bb147152-8c96-4e6d-b982-2e6076cc6145,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:55.971149 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ca673945-bf9d-438a-9eb7-c9a9ced1da47 req-2668aa0e-e63a-446b-bcdb-c008f42385d4 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2475193 May 24 11:18:55.975870 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:55.976750 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device ff3663f5-a200-4bed-988a-f47af0d746d7 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:18:56.011021 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:18:56.026665 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:56.032375 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-3522c421-8843-46ea-9e22-3933bb231c21 admin admin] 10.222.0.18 "PUT /v2.0/routers/d37dc0a7-5e53-4516-8020-b2990086e8c5/add_router_interface HTTP/1.1" status: 200 len: 503 time: 6.1044760 May 24 11:18:56.075742 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-ccbb0f25-23fa-4025-abec-df5d4ea591d3 None None] Port 1a4dddfd-3c51-421f-b942-92a179e987b8 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:18:56.077898 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:56.095046 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9233a97a-5cf6-4529-8a81-39073d46adae', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'bb147152-8c96-4e6d-b982-2e6076cc6145', 'code': 200} May 24 11:18:56.101114 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a331a856-6b58-4ca9-8b9a-3a92a0febd0c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:56.128832 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Sync routers for ids ['d37dc0a7-5e53-4516-8020-b2990086e8c5'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:18:56.160132 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Routers {'d37dc0a7-5e53-4516-8020-b2990086e8c5'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:18:56.161454 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['d37dc0a7-5e53-4516-8020-b2990086e8c5'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:56.222016 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:56.337410 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8ceb25d3-0554-4d9e-85a8-948b5e378954 req-22346314-67e8-4262-b10c-946fe0f4d91f service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4034524 May 24 11:18:56.407800 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Set router d37dc0a7-5e53-4516-8020-b2990086e8c5 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:18:56.412144 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-73fdefc9-b776-4a86-8477-8caa35f795d4 req-68263c22-5c2a-4113-b552-b6904d3f4286 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:56.413746 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-73fdefc9-b776-4a86-8477-8caa35f795d4 req-68263c22-5c2a-4113-b552-b6904d3f4286 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 946 time: 0.3639882 May 24 11:18:56.515098 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=bb147152-8c96-4e6d-b982-2e6076cc6145,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:56.581119 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.554s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:56.614809 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-73fdefc9-b776-4a86-8477-8caa35f795d4 req-84a34ea4-9120-4998-9bd5-2e47745ed241 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1936321 May 24 11:18:56.627216 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:56.643654 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:56.662300 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b4ebe1e6-d36f-49af-85cb-37f19930cc52 req-91a6266a-9cd1-4197-be78-8c56cf657c8a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:56.666390 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b4ebe1e6-d36f-49af-85cb-37f19930cc52 req-91a6266a-9cd1-4197-be78-8c56cf657c8a service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 1099 time: 0.2102106 May 24 11:18:56.761943 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=4e43bacd-47d7-4c30-b3b3-64bc8458a6dc,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:56.774489 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:56.787941 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b4ebe1e6-d36f-49af-85cb-37f19930cc52 req-3733f1e9-1325-45fd-b5e5-a172f1872521 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1095254 May 24 11:18:56.789014 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:56.834859 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Notify callbacks [] for port, precommit_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:56.871238 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-7bca9cf0-fe51-4c9a-b63a-722c6dbd50c2 req-9c3549f5-3b13-4f88-89e3-cd954f0a824a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:56.886641 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:56.899848 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Transition to ACTIVE for port object e648ddfe-6f08-4b5c-8c21-e6794aa56080 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:56.933217 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'name': 'tempest-TestMinimumBasicScenario-648060998-router', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181'}], 'enable_snat': True}, 'gw_port_id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:44Z', 'updated_at': '2021-05-24T11:18:53Z', 'revision_number': 4, 'project_id': '675d4e36e7c14b46864dd445f05bc0ec', 'gw_port': {'id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:04:4a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 385, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:46Z', 'updated_at': '2021-05-24T11:18:52Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '1a4dddfd-3c51-421f-b942-92a179e987b8', 'name': '', 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'mac_address': 'fa:16:3e:cf:39:ff', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'device_owner': 'network:router_interface', 'standard_attr_id': 390, 'fixed_ips': [{'subnet_id': 'a90ff9ef-eadb-4731-b071-d0c9fe13cb00', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:51Z', 'updated_at': '2021-05-24T11:18:53Z', 'revision_number': 2, 'project_id': '675d4e36e7c14b46864dd445f05bc0ec', 'subnets': [{'id': 'a90ff9ef-eadb-4731-b071-d0c9fe13cb00', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:56.939923 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Checking router: d37dc0a7-5e53-4516-8020-b2990086e8c5 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:18:56.941752 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Updating router d37dc0a7-5e53-4516-8020-b2990086e8c5 port 1a4dddfd-3c51-421f-b942-92a179e987b8 binding host dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:18:56.965431 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:56.966885 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9c3549f5-3b13-4f88-89e3-cd954f0a824a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Pushing event deleted for resources: {'Port': ['ID=ff3663f5-a200-4bed-988a-f47af0d746d7,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:56.971398 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9c3549f5-3b13-4f88-89e3-cd954f0a824a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:57.021182 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=4e43bacd-47d7-4c30-b3b3-64bc8458a6dc,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:57.029466 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.241s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:57.030069 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.054s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:57.069842 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:57.108415 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:57.146929 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7f2f3b0-90a9-486b-bdd8-b9ac5d0db881 req-d489240a-5658-455f-b6b2-93874268d86c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:57.147196 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 23bbb434-e051-4fbc-8e80-f213cd1ab793 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:57.162189 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a7f2f3b0-90a9-486b-bdd8-b9ac5d0db881 req-d489240a-5658-455f-b6b2-93874268d86c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.5917637 May 24 11:18:57.174191 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Attempting to bind port e648ddfe-6f08-4b5c-8c21-e6794aa56080 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:18:57.174756 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Attempting to bind port e648ddfe-6f08-4b5c-8c21-e6794aa56080 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:18:57.175188 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Attempting to bind port e648ddfe-6f08-4b5c-8c21-e6794aa56080 on network 858b8fad-a73f-4578-9079-6e731b9ff7e2 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:57.197849 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-7bca9cf0-fe51-4c9a-b63a-722c6dbd50c2 req-9c3549f5-3b13-4f88-89e3-cd954f0a824a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] No DVR routers for this DVR port ff3663f5-a200-4bed-988a-f47af0d746d7 on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:18:57.209215 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:18:57.209593 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Checking segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:57.210003 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:57.210406 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Bound using segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:18:57.210906 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Bound port: e648ddfe-6f08-4b5c-8c21-e6794aa56080, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:18:57.211409 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:57.265204 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-7bca9cf0-fe51-4c9a-b63a-722c6dbd50c2 req-9c3549f5-3b13-4f88-89e3-cd954f0a824a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:57.271261 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-597a084c-17d6-4f49-b1d0-a6d250776b93 None None] Pushing event updated for resources: {'Port': ['ID=e648ddfe-6f08-4b5c-8c21-e6794aa56080,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:57.288532 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a7f2f3b0-90a9-486b-bdd8-b9ac5d0db881 req-b270bdba-364a-4af6-930d-d85bb6e58cd7 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1254120 May 24 11:18:57.289688 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '78ccab16-0ff0-4b84-8d91-71e4092110e9', 'name': 'network-vif-deleted', 'tag': 'ff3663f5-a200-4bed-988a-f47af0d746d7'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:18:57.291151 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-597a084c-17d6-4f49-b1d0-a6d250776b93 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.261s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:57.357907 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7bca9cf0-fe51-4c9a-b63a-722c6dbd50c2 req-9c3549f5-3b13-4f88-89e3-cd954f0a824a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.93 "DELETE /v2.0/ports/ff3663f5-a200-4bed-988a-f47af0d746d7 HTTP/1.1" status: 204 len: 173 time: 2.6024625 May 24 11:18:57.359593 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Allocated IP 172.20.0.1 (be65d1af-351e-4e5b-8c19-1b04ff5cfc29/994391be-baae-4f60-9ef9-bf02d19d3477/5646b226-211d-4c06-b7f8-5dfc132e817e) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:18:57.359839 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f7945b43-d57f-4bd3-8ac5-9eb5e0ec25f1 req-86cffcd4-434c-4749-8494-b46a3f898ee5 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=78ccab16-0ff0-4b84-8d91-71e4092110e9 HTTP/1.1" status: 200 len: 210 time: 0.2820220 May 24 11:18:57.377984 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] For port e648ddfe-6f08-4b5c-8c21-e6794aa56080, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:18:57.403019 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] For port e648ddfe-6f08-4b5c-8c21-e6794aa56080, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=e648ddfe-6f08-4b5c-8c21-e6794aa56080,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:18:57.404298 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 23bbb434-e051-4fbc-8e80-f213cd1ab793 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:57.418397 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 23bbb434-e051-4fbc-8e80-f213cd1ab793 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:57.418777 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:57.477825 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:57.499606 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:57.522746 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '78ccab16-0ff0-4b84-8d91-71e4092110e9', 'name': 'network-vif-deleted', 'tag': 'ff3663f5-a200-4bed-988a-f47af0d746d7', 'status': 'completed', 'code': 200} May 24 11:18:57.675034 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:18:57.675790 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:57.792175 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:57.813067 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Notify callbacks [] for port, precommit_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:57.830075 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.provisioning_blocks [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Transition to ACTIVE for port object 1a4dddfd-3c51-421f-b942-92a179e987b8 will not be triggered until provisioned by entity L2. {{(pid=91822) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:57.836822 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-64f5ebe7-dbc9-402c-85b8-3f6510851302 req-6f2d3e7c-ae1c-43b0-b69a-71e4f8f7a200 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:57.846728 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-64f5ebe7-dbc9-402c-85b8-3f6510851302 req-6f2d3e7c-ae1c-43b0-b69a-71e4f8f7a200 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.2540190 May 24 11:18:57.857057 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:57.866230 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-099a4a1a-0c7d-4e78-b27a-03391453046d None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:57.948892 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Transition to ACTIVE for port object 5646b226-211d-4c06-b7f8-5dfc132e817e will not be triggered until provisioned by entity DHCP. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:18:57.960530 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:18:57.985823 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-64f5ebe7-dbc9-402c-85b8-3f6510851302 req-99ebee33-a5d9-42bc-a33d-03f78478fed1 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1216457 May 24 11:18:58.051874 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:58.052446 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-903d5b5f-9a25-4901-acfc-c502406715e8 req-d311f0ba-67ee-4d30-a9f1-128d8d5aff74 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:58.067407 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-903d5b5f-9a25-4901-acfc-c502406715e8 req-d311f0ba-67ee-4d30-a9f1-128d8d5aff74 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 946 time: 0.1919727 May 24 11:18:58.077214 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-233b0fa5-a780-462c-b38d-c7f2de17371d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:58.089264 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-597a084c-17d6-4f49-b1d0-a6d250776b93 None None] Pushing event updated for resources: {'Port': ['ID=e648ddfe-6f08-4b5c-8c21-e6794aa56080,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:58.097816 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f6f5c39-61be-4156-8d1a-d54d91752876 None None] Pushing event updated for resources: {'Port': ['ID=1a4dddfd-3c51-421f-b942-92a179e987b8,revision_number=3']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:58.179118 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-903d5b5f-9a25-4901-acfc-c502406715e8 req-3af74de4-7070-4dc5-bf4e-714186b686d6 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1090271 May 24 11:18:58.220106 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:18:58.224487 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:18:58.250774 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:18:58.364914 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Pushing event updated for resources: {'Port': ['ID=5646b226-211d-4c06-b7f8-5dfc132e817e,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:58.369484 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.292s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:58.378255 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-597a084c-17d6-4f49-b1d0-a6d250776b93 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.877s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:58.396746 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-1f6f5c39-61be-4156-8d1a-d54d91752876 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.530s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:58.411566 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-687570'] for router_interface, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:58.580253 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Attempting to bind port 1a4dddfd-3c51-421f-b942-92a179e987b8 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:18:58.580828 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Attempting to bind port 1a4dddfd-3c51-421f-b942-92a179e987b8 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '88ac69f8-b455-4003-825b-2df6e05e49fa', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 531, 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a'}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:18:58.581454 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Attempting to bind port 1a4dddfd-3c51-421f-b942-92a179e987b8 on network 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:18:58.591638 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:18:58.592399 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Checking segment: {'id': '88ac69f8-b455-4003-825b-2df6e05e49fa', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 531, 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91822) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:18:58.593062 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:58.593832 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Bound using segment: {'id': '88ac69f8-b455-4003-825b-2df6e05e49fa', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 531, 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a'} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:18:58.594867 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Bound port: 1a4dddfd-3c51-421f-b942-92a179e987b8, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '88ac69f8-b455-4003-825b-2df6e05e49fa', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 531, 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a'}}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:18:58.595438 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:58.872373 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] For port 1a4dddfd-3c51-421f-b942-92a179e987b8, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91822) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:18:58.875980 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] For port 1a4dddfd-3c51-421f-b942-92a179e987b8, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=1a4dddfd-3c51-421f-b942-92a179e987b8,segment=NetworkSegment(88ac69f8-b455-4003-825b-2df6e05e49fa),segment_id=88ac69f8-b455-4003-825b-2df6e05e49fa)] {{(pid=91822) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:18:58.885345 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40b8db66-fc48-482c-bf48-a57dc9fa6d79 req-2cdce16a-7781-482f-a8cb-3bd0506e830f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:58.900585 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:58.912874 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-7a441d98-55ac-4718-8ca8-7d416931d61c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:58.932858 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-40b8db66-fc48-482c-bf48-a57dc9fa6d79 req-2cdce16a-7781-482f-a8cb-3bd0506e830f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.3646266 May 24 11:18:59.034723 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:59.037059 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:59.045874 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Routers {'d37dc0a7-5e53-4516-8020-b2990086e8c5'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:18:59.048605 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['d37dc0a7-5e53-4516-8020-b2990086e8c5'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:59.067997 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-40b8db66-fc48-482c-bf48-a57dc9fa6d79 req-dc4a472b-fa09-4f81-a20f-c9bf3853e4ee tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1338129 May 24 11:18:59.100039 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:59.130163 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 23bbb434-e051-4fbc-8e80-f213cd1ab793, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=23bbb434-e051-4fbc-8e80-f213cd1ab793,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:18:59.147709 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:59.150149 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f6f5c39-61be-4156-8d1a-d54d91752876 None None] Pushing event updated for resources: {'Port': ['ID=1a4dddfd-3c51-421f-b942-92a179e987b8,revision_number=4']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:59.153997 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:18:59.162162 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-1f6f5c39-61be-4156-8d1a-d54d91752876 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.249s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:59.237950 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:51Z', 'revision_number': 3, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:18:57Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:18:59.247774 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8424f512-d740-4d9c-8b12-c606c60f928d None None] The sync data for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi is: [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:51Z', 'revision_number': 3, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:18:57Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:18:59.315935 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-d3540b95-4f7c-4f5a-b4c8-a619366b74c0 None None] Provisioning for port 5646b226-211d-4c06-b7f8-5dfc132e817e completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:18:59.338985 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-d3540b95-4f7c-4f5a-b4c8-a619366b74c0 None None] Provisioning complete for port 5646b226-211d-4c06-b7f8-5dfc132e817e triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:59.338985 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d3540b95-4f7c-4f5a-b4c8-a619366b74c0 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:59.468735 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:18:59.582474 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] Set router d37dc0a7-5e53-4516-8020-b2990086e8c5 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:18:59.584334 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=23bbb434-e051-4fbc-8e80-f213cd1ab793,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:18:59.588923 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.436s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:18:59.622877 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-036be8a5-ef04-421c-8912-9e0a19face80 req-a2dd91ed-8de4-4b44-b890-d09e293b85e0 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:59.634560 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-a2dd91ed-8de4-4b44-b890-d09e293b85e0 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/networks?id=a21e7e1e-54f7-4844-b98d-f658bdb26a4c HTTP/1.1" status: 200 len: 809 time: 0.5072215 May 24 11:18:59.657645 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-af93d31c-8999-4f2f-a2ea-f068e6095860 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/quotas/7c9e587a510c4777ab57643ed31ea73f HTTP/1.1" status: 200 len: 375 time: 0.0221884 May 24 11:18:59.661426 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:18:59.661871 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:18:59.750565 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-dbc239a3-ca38-46dc-9c99-2daaf4a3d260 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=7c9e587a510c4777ab57643ed31ea73f&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0846272 May 24 11:18:59.786639 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-d3540b95-4f7c-4f5a-b4c8-a619366b74c0 None None] Port 5646b226-211d-4c06-b7f8-5dfc132e817e cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:18:59.948539 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55237290-2a2b-4a6a-ade5-11ab3d9f0fd4 req-73fbb11b-9bfa-44cc-8617-baa269f0fa8a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:18:59.951962 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-55237290-2a2b-4a6a-ade5-11ab3d9f0fd4 req-73fbb11b-9bfa-44cc-8617-baa269f0fa8a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.4838858 May 24 11:18:59.996548 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-93ecf4c8-687d-4e52-b468-e6584d484129 service neutron] 10.222.0.18 "GET /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2383220 May 24 11:19:00.005415 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:00.017463 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:00.088702 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e37af2f4-f9f0-4645-9d54-237ddc0a9cc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:00.254987 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-55237290-2a2b-4a6a-ade5-11ab3d9f0fd4 req-cf9c6048-3fca-4987-9122-924929fe00df tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2951763 May 24 11:19:00.259729 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-e7b8bb64-9c66-4598-b720-8d92d4961f35 service neutron] 10.222.0.18 "GET /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2620554 May 24 11:19:00.301925 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'name': 'tempest-TestMinimumBasicScenario-648060998-router', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181'}], 'enable_snat': True}, 'gw_port_id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:44Z', 'updated_at': '2021-05-24T11:18:53Z', 'revision_number': 4, 'project_id': '675d4e36e7c14b46864dd445f05bc0ec', 'gw_port': {'id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:04:4a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 385, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:46Z', 'updated_at': '2021-05-24T11:18:59Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '1a4dddfd-3c51-421f-b942-92a179e987b8', 'name': '', 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'mac_address': 'fa:16:3e:cf:39:ff', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'device_owner': 'network:router_interface', 'standard_attr_id': 390, 'fixed_ips': [{'subnet_id': 'a90ff9ef-eadb-4731-b071-d0c9fe13cb00', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:51Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '675d4e36e7c14b46864dd445f05bc0ec', 'subnets': [{'id': 'a90ff9ef-eadb-4731-b071-d0c9fe13cb00', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:00.303404 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Pushing event updated for resources: {'Port': ['ID=5646b226-211d-4c06-b7f8-5dfc132e817e,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:00.307333 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.219s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:00.312309 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d147be67-2b16-4c8a-9375-c1334a3a8617 None None] The sync data for ids ['d37dc0a7-5e53-4516-8020-b2990086e8c5'] in dv-cinder-784092-9-iscsi is: [{'id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'name': 'tempest-TestMinimumBasicScenario-648060998-router', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181'}], 'enable_snat': True}, 'gw_port_id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:44Z', 'updated_at': '2021-05-24T11:18:53Z', 'revision_number': 4, 'project_id': '675d4e36e7c14b46864dd445f05bc0ec', 'gw_port': {'id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:04:4a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 385, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:46Z', 'updated_at': '2021-05-24T11:18:59Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '1a4dddfd-3c51-421f-b942-92a179e987b8', 'name': '', 'network_id': '9672fa3f-743d-4d68-8dd2-0bf684ca5e6a', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'mac_address': 'fa:16:3e:cf:39:ff', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'device_owner': 'network:router_interface', 'standard_attr_id': 390, 'fixed_ips': [{'subnet_id': 'a90ff9ef-eadb-4731-b071-d0c9fe13cb00', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:51Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '675d4e36e7c14b46864dd445f05bc0ec', 'subnets': [{'id': 'a90ff9ef-eadb-4731-b071-d0c9fe13cb00', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:19:00.329370 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-533332', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-687487'] for router_interface, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:00.987649 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5def54c8-9eee-408f-b92e-125bf75093dd req-ca1e1802-cdcc-499b-88c3-c9c393847c79 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:00.990233 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5def54c8-9eee-408f-b92e-125bf75093dd req-ca1e1802-cdcc-499b-88c3-c9c393847c79 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.2613661 May 24 11:19:01.022070 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dda4d75e-7376-4898-8b7f-353890f7692a req-b3ce0923-ae8c-4ccb-bc72-37a9d8678fa8 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:01.026731 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dda4d75e-7376-4898-8b7f-353890f7692a req-b3ce0923-ae8c-4ccb-bc72-37a9d8678fa8 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 946 time: 0.3472807 May 24 11:19:01.106708 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dda4d75e-7376-4898-8b7f-353890f7692a req-a4734caa-5303-49cc-b7ff-708bca44bd25 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0730219 May 24 11:19:01.108685 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5def54c8-9eee-408f-b92e-125bf75093dd req-23c4512d-5e1c-4a53-8c83-75c882af5208 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1052544 May 24 11:19:01.144567 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:19:01.146417 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:19:01.568817 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-69ecb463-255f-4b3f-8038-0346c9de54f1 req-603db722-d4f9-402e-89ce-7629c1eb6bae tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 210 time: 0.2892063 May 24 11:19:01.744809 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 23bbb434-e051-4fbc-8e80-f213cd1ab793 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:01.868524 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Router 81705dbf-841d-45b4-8b2d-afa9886a8e59 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:19:01.976056 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-8b2735a9-9b7e-4798-a7af-c81a87e7a37e None None] Provisioning complete for port 5646b226-211d-4c06-b7f8-5dfc132e817e triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:01.976523 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8b2735a9-9b7e-4798-a7af-c81a87e7a37e None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:02.044266 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 23bbb434-e051-4fbc-8e80-f213cd1ab793 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:02.044643 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:02.135748 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-8b2735a9-9b7e-4798-a7af-c81a87e7a37e None None] Port 5646b226-211d-4c06-b7f8-5dfc132e817e cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:19:02.433585 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device e648ddfe-6f08-4b5c-8c21-e6794aa56080 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:02.514672 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-04f3f156-2e95-490c-95b8-f46b670fd5d7 req-d27a5393-2189-4fb5-9f5b-b8ac8d0e4bc1 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:02.520816 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-04f3f156-2e95-490c-95b8-f46b670fd5d7 req-d27a5393-2189-4fb5-9f5b-b8ac8d0e4bc1 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.7153616 May 24 11:19:02.591018 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3f9e2af9-2748-48f3-aa35-01ab06f72cd5 req-73471314-8ad0-4918-bf55-d74be2959f7f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:02.600568 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port e648ddfe-6f08-4b5c-8c21-e6794aa56080 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:19:02.602065 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3f9e2af9-2748-48f3-aa35-01ab06f72cd5 req-73471314-8ad0-4918-bf55-d74be2959f7f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.2024574 May 24 11:19:02.613765 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port e648ddfe-6f08-4b5c-8c21-e6794aa56080 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:02.614274 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:02.646325 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:19:02.658288 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] 10.222.0.18 "PUT /v2.0/routers/81705dbf-841d-45b4-8b2d-afa9886a8e59/add_router_interface HTTP/1.1" status: 200 len: 503 time: 8.7313995 May 24 11:19:02.695604 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3f9e2af9-2748-48f3-aa35-01ab06f72cd5 req-4ed76856-ea5a-4cb5-94a1-b45b7e149bca tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0922446 May 24 11:19:02.741874 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-04f3f156-2e95-490c-95b8-f46b670fd5d7 req-e2055e11-a52a-4944-9045-c38fee5914e2 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2079742 May 24 11:19:03.023485 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:03.082449 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port e648ddfe-6f08-4b5c-8c21-e6794aa56080, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=e648ddfe-6f08-4b5c-8c21-e6794aa56080,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:03.092333 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:03.093422 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 1a4dddfd-3c51-421f-b942-92a179e987b8 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:03.185205 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9477b31b-3846-413a-89ac-1891a067984b req-188a6f3f-e9c8-4164-8a65-8e31200e79dd tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 210 time: 0.1379106 May 24 11:19:03.224337 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:03.313327 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Sync routers for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:19:03.335556 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:19:03.336582 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:03.447550 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-036be8a5-ef04-421c-8912-9e0a19face80 req-13280f61-2ea1-493f-b1be-d83695e8074c tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:03.450022 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-13280f61-2ea1-493f-b1be-d83695e8074c tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.93 "GET /v2.0/networks?id=a21e7e1e-54f7-4844-b98d-f658bdb26a4c HTTP/1.1" status: 200 len: 809 time: 0.4163630 May 24 11:19:03.464180 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Request body: {'port': {'device_id': '07f504cc-a04d-4738-8be7-ff333195f9d9', 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c', 'admin_state_up': True, 'tenant_id': '7c9e587a510c4777ab57643ed31ea73f'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:03.488391 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 1a4dddfd-3c51-421f-b942-92a179e987b8 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:19:03.512749 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=e648ddfe-6f08-4b5c-8c21-e6794aa56080,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:03.542112 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.318s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:03.547068 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 1a4dddfd-3c51-421f-b942-92a179e987b8 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:03.547920 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:03.755027 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:19:03.782427 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:03.798427 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Usage tracker for resource:port and tenant:7c9e587a510c4777ab57643ed31ea73f is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:03.804044 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Quota usage for port was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:03.827807 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:03.855026 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Made reservation on behalf of 7c9e587a510c4777ab57643ed31ea73f for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:19:03.861819 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:03.927955 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:04.066430 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56422) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:04.080232 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Request body: {'network': {'name': 'tempest-TestVolumeBootPattern-1369558850-network', 'tenant_id': '672c5f4985084a8492a86667aceb0768'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:04.091061 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:04.096110 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Usage tracker for resource:network and tenant:672c5f4985084a8492a86667aceb0768 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:04.100254 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:04.108308 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:04.131100 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Made reservation on behalf of 672c5f4985084a8492a86667aceb0768 for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:19:04.134207 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:04.143002 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-86754eb7-ce13-464a-bfb7-c3768ee0d9db req-08209550-9234-4e89-8579-09639f1c87db tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:04.146496 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-86754eb7-ce13-464a-bfb7-c3768ee0d9db req-08209550-9234-4e89-8579-09639f1c87db tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1743424 May 24 11:19:04.162657 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:04.211132 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:04.221710 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Usage tracker for resource:security_group_rule and tenant:672c5f4985084a8492a86667aceb0768 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:04.227574 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:04.258367 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 1a4dddfd-3c51-421f-b942-92a179e987b8, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=1a4dddfd-3c51-421f-b942-92a179e987b8,segment=NetworkSegment(88ac69f8-b455-4003-825b-2df6e05e49fa),segment_id=88ac69f8-b455-4003-825b-2df6e05e49fa)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:04.274664 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:04.278745 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:04.298934 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-86754eb7-ce13-464a-bfb7-c3768ee0d9db req-3ee2cf75-7b98-4fe0-ad6c-37b44e5b78de tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1465502 May 24 11:19:04.341504 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:04.397876 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Allocated IP 172.20.0.10 (a21e7e1e-54f7-4844-b98d-f658bdb26a4c/1ed78d41-8564-48a2-acc1-2ef21df88d9e/11c2ccd2-652a-4862-8c69-587b35b23237) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:19:04.422915 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=1a4dddfd-3c51-421f-b942-92a179e987b8,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:04.429861 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.151s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:04.490274 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device e648ddfe-6f08-4b5c-8c21-e6794aa56080 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:04.642528 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port e648ddfe-6f08-4b5c-8c21-e6794aa56080 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:04.642946 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:04.703655 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c53514e1-8f99-4e92-ad5e-f1c3407779d8 req-b07a698f-fd5c-4580-89b3-0fa76af8b931 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:04.711920 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c53514e1-8f99-4e92-ad5e-f1c3407779d8 req-b07a698f-fd5c-4580-89b3-0fa76af8b931 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.2138634 May 24 11:19:04.735482 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:04.747592 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:04.758205 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-32bfb5c4-43a8-4fca-89be-23b335841d3e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:04.771844 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c53514e1-8f99-4e92-ad5e-f1c3407779d8 req-10ec811d-a149-497d-abb3-d85c112bf42d tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0571685 May 24 11:19:04.812724 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:18:59Z', 'revision_number': 2, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:04.845585 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Checking router: 81705dbf-841d-45b4-8b2d-afa9886a8e59 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:19:04.855505 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Updating router 81705dbf-841d-45b4-8b2d-afa9886a8e59 port 5646b226-211d-4c06-b7f8-5dfc132e817e binding host dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:19:04.872460 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] vlan segment allocate from pool success with {'vlan_id': 520, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:19:04.893502 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-b2fc1475-a56a-4635-99c5-17aaaaa6ae08 req-26d05384-ba76-42f4-8985-a16b78ec27ff tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Synchronizing usage tracker for tenant:7c9e587a510c4777ab57643ed31ea73f on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:04.944462 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-b2fc1475-a56a-4635-99c5-17aaaaa6ae08 req-26d05384-ba76-42f4-8985-a16b78ec27ff tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Unset dirty status for tenant:7c9e587a510c4777ab57643ed31ea73f on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:04.958959 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b2fc1475-a56a-4635-99c5-17aaaaa6ae08 req-26d05384-ba76-42f4-8985-a16b78ec27ff tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 210 time: 0.1452312 May 24 11:19:04.960411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=160a5414-06db-4f2b-92ba-2a3a3924021e,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:04.974169 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:04.974685 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Added segment c1fbdc79-842c-49eb-8789-2a1de3ab2b9f of type vlan for network e6b6c00d-d044-4bc7-92ce-02e445f1f072 May 24 11:19:05.002222 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:05.002947 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:05.004286 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:05.004847 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:05.005417 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:05.005968 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:05.006470 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:05.007264 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:05.007795 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:05.008482 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:05.008986 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:05.009810 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:05.021692 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.263s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:05.037151 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:05.067150 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:05.098269 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:05.119506 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:05.123504 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:05.124538 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:05.151566 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:05.225127 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:05.590938 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Pushing event updated for resources: {'Network': ['ID=e6b6c00d-d044-4bc7-92ce-02e445f1f072,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:05.605532 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 885 time: 1.5365891 May 24 11:19:05.610398 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 56440) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:05.617219 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.466s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:05.624709 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9c3549f5-3b13-4f88-89e3-cd954f0a824a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.036s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:05.626483 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Request body: {'subnet': {'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestVolumeBootPattern-1369558850-subnet', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'ip_version': 4}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:05.631772 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:05.635966 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Usage tracker for resource:subnet and tenant:672c5f4985084a8492a86667aceb0768 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:05.640180 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:05.647929 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:05.652857 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network e6b6c00d-d044-4bc7-92ce-02e445f1f072 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:19:05.667027 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Made reservation on behalf of 672c5f4985084a8492a86667aceb0768 for: {'subnet': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:19:05.670051 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-535456753'] for subnet, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:05.711543 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Network a21e7e1e-54f7-4844-b98d-f658bdb26a4c is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:19:05.712417 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Pushing event updated for resources: {'Port': ['ID=11c2ccd2-652a-4862-8c69-587b35b23237,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:05.716457 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:05.727524 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:19:05.730336 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:19:05.831473 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Notify callbacks [] for port, precommit_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:05.834379 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:05.835274 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:05.869532 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Transition to ACTIVE for port object 5646b226-211d-4c06-b7f8-5dfc132e817e will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:19:05.869775 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-14184f00-8d42-4e1c-b45c-458b1fcf4b10 req-2864db12-5982-4ad9-acd6-a2464c6b55a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:05.871962 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-14184f00-8d42-4e1c-b45c-458b1fcf4b10 req-2864db12-5982-4ad9-acd6-a2464c6b55a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.3364074 May 24 11:19:05.888311 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 2.4278753 May 24 11:19:05.889471 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:05.897555 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-0972f257-375d-43cb-94f9-cfccada8da58 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:05.911406 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Request body: {'port': {'device_id': '07f504cc-a04d-4738-8be7-ff333195f9d9', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:05.995007 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2df3e20d-8580-4abb-b562-ae817939b96f req-4fcd88f4-d3f9-44c8-88ee-4e54776f3448 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:05.996229 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-14184f00-8d42-4e1c-b45c-458b1fcf4b10 req-48a17a92-9fef-4525-9c12-cc9f1583fb0a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1123457 May 24 11:19:06.012980 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2df3e20d-8580-4abb-b562-ae817939b96f req-4fcd88f4-d3f9-44c8-88ee-4e54776f3448 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=b303f83a84794e5ba6a1bab9c76b429c&device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 1099 time: 0.6303315 May 24 11:19:06.023098 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1ff26346-8675-43df-b264-74dc1c913b76 None None] Pushing event updated for resources: {'Port': ['ID=5646b226-211d-4c06-b7f8-5dfc132e817e,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:06.034400 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-1ff26346-8675-43df-b264-74dc1c913b76 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.138s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:06.091963 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:06.456777 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 1a4dddfd-3c51-421f-b942-92a179e987b8 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:06.539748 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Attempting to bind port 5646b226-211d-4c06-b7f8-5dfc132e817e on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:19:06.540157 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Attempting to bind port 5646b226-211d-4c06-b7f8-5dfc132e817e by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:19:06.540640 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Attempting to bind port 5646b226-211d-4c06-b7f8-5dfc132e817e on network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:19:06.569274 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:19:06.569628 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Checking segment: {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:19:06.569924 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:06.570522 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Bound using segment: {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:19:06.571182 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Bound port: 5646b226-211d-4c06-b7f8-5dfc132e817e, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:19:06.571641 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:06.601664 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2df3e20d-8580-4abb-b562-ae817939b96f req-b5299e08-b22f-4ec0-a329-4473550b8a96 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:06.603220 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2df3e20d-8580-4abb-b562-ae817939b96f req-b5299e08-b22f-4ec0-a329-4473550b8a96 service neutron] 10.222.0.93 "GET /v2.0/networks?id=1d92641a-d8f9-4ed6-9abd-f464bb26811d HTTP/1.1" status: 200 len: 912 time: 0.5582845 May 24 11:19:06.612713 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fca9f844-66fb-4487-b2f3-285465b8028f req-15c38001-bca1-4839-b6fe-6a6d6e76819f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:06.620830 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-fca9f844-66fb-4487-b2f3-285465b8028f req-15c38001-bca1-4839-b6fe-6a6d6e76819f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 1.0920546 May 24 11:19:06.686693 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d23ac887-8e44-4b39-bb2e-f4fc6b0a4e43 req-c96c83a3-8191-4b07-a112-b7b89423a5f5 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:06.688510 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d23ac887-8e44-4b39-bb2e-f4fc6b0a4e43 req-c96c83a3-8191-4b07-a112-b7b89423a5f5 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 932 time: 0.2073791 May 24 11:19:06.727650 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] For port 5646b226-211d-4c06-b7f8-5dfc132e817e, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:19:06.731711 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] For port 5646b226-211d-4c06-b7f8-5dfc132e817e, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=5646b226-211d-4c06-b7f8-5dfc132e817e,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:19:06.740212 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:06.749889 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:06.764260 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d23ac887-8e44-4b39-bb2e-f4fc6b0a4e43 req-cbe3d19a-26e5-442c-8ec0-56c3e320050a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0677583 May 24 11:19:06.818594 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-fca9f844-66fb-4487-b2f3-285465b8028f req-24779088-69df-4be7-87a8-9a90bae26db2 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1964967 May 24 11:19:06.835314 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 1a4dddfd-3c51-421f-b942-92a179e987b8 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:06.836158 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:06.853581 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2df3e20d-8580-4abb-b562-ae817939b96f req-bed64747-ff03-4e2c-ab6f-7777f35e4072 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=eab44150-44f7-4194-9b76-436d421794e8 HTTP/1.1" status: 200 len: 1001 time: 0.2117517 May 24 11:19:06.859627 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 11c2ccd2-652a-4862-8c69-587b35b23237 {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:19:06.875286 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:06.904041 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Transition to ACTIVE for port object 11c2ccd2-652a-4862-8c69-587b35b23237 will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:19:06.910561 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:06.916047 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4d50c9ce-7ff0-4ff5-96fc-7ea4514d105f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:06.959363 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2df3e20d-8580-4abb-b562-ae817939b96f req-a8206d78-5d5e-4047-a0ed-28a7bd7c8b7b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:06.960777 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2df3e20d-8580-4abb-b562-ae817939b96f req-a8206d78-5d5e-4047-a0ed-28a7bd7c8b7b service neutron] 10.222.0.93 "GET /v2.0/subnets?id=2fc4be2e-875c-4d10-b857-a48f90c53cf2 HTTP/1.1" status: 200 len: 850 time: 0.0950840 May 24 11:19:07.203908 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:19:07.212324 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:07.326067 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-534788555', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-535456743'] for subnet, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:07.343817 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:07.412179 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Pushing event updated for resources: {'Subnet': ['ID=a25edff7-c0c6-44d4-ad1c-a38e28a26a35,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:07.426250 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:07.426624 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:07.426981 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:07.427335 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:07.427680 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:07.428093 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:07.428437 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:07.428780 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:07.429573 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:07.430055 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:07.430401 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:07.430873 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:07.446250 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Pushing event updated for resources: {'Port': ['ID=11c2ccd2-652a-4862-8c69-587b35b23237,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:07.447939 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:07.512404 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Checking segment: {'id': 'c1fbdc79-842c-49eb-8789-2a1de3ab2b9f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 520, 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:19:07.513596 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Network hostable DHCP agents. Network: e6b6c00d-d044-4bc7-92ce-02e445f1f072, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91820) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:19:07.529284 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2df3e20d-8580-4abb-b562-ae817939b96f req-81b501c1-70c1-4249-8985-8804957a2516 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:07.536263 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2df3e20d-8580-4abb-b562-ae817939b96f req-81b501c1-70c1-4249-8985-8804957a2516 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=1d92641a-d8f9-4ed6-9abd-f464bb26811d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.5110354 May 24 11:19:07.562914 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.646s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:07.587971 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Attempting to bind port 11c2ccd2-652a-4862-8c69-587b35b23237 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:19:07.589215 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Attempting to bind port 11c2ccd2-652a-4862-8c69-587b35b23237 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': 'f3295bfe-6894-409e-b0cc-96311bd90247', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 514, 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:19:07.589809 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Attempting to bind port 11c2ccd2-652a-4862-8c69-587b35b23237 on network a21e7e1e-54f7-4844-b98d-f658bdb26a4c {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:19:07.609380 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Port 11c2ccd2-652a-4862-8c69-587b35b23237 on network a21e7e1e-54f7-4844-b98d-f658bdb26a4c not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:19:07.610733 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Attempting to bind port 11c2ccd2-652a-4862-8c69-587b35b23237 on network a21e7e1e-54f7-4844-b98d-f658bdb26a4c {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:19:07.636833 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 52), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:19:07.637593 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Checking segment: {'id': 'f3295bfe-6894-409e-b0cc-96311bd90247', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 514, 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:19:07.638411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Bound using segment: {'id': 'f3295bfe-6894-409e-b0cc-96311bd90247', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 514, 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:19:07.639546 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Bound port: 11c2ccd2-652a-4862-8c69-587b35b23237, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'f3295bfe-6894-409e-b0cc-96311bd90247', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 514, 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:19:07.640134 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:07.719126 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2df3e20d-8580-4abb-b562-ae817939b96f req-ed28530b-9091-4e14-b3fa-1a451d9b7f07 service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1729710 May 24 11:19:07.751542 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] For port 11c2ccd2-652a-4862-8c69-587b35b23237, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:19:07.755894 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] For port 11c2ccd2-652a-4862-8c69-587b35b23237, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=11c2ccd2-652a-4862-8c69-587b35b23237,segment=NetworkSegment(f3295bfe-6894-409e-b0cc-96311bd90247),segment_id=f3295bfe-6894-409e-b0cc-96311bd90247)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:19:07.771831 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f47c3763-8516-4735-8160-2d19a43c31e4 req-233ea8bc-ef07-432a-8892-1de174e73c7b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:07.779475 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f47c3763-8516-4735-8160-2d19a43c31e4 req-233ea8bc-ef07-432a-8892-1de174e73c7b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.4231458 May 24 11:19:07.797860 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:07.806192 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-38e2c2bb-5a29-45b9-8992-7d45483e0658 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:07.896487 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:07.897512 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:07.899495 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] 10.222.0.93 "PUT /v2.0/ports/11c2ccd2-652a-4862-8c69-587b35b23237 HTTP/1.1" status: 200 len: 1094 time: 1.9936099 May 24 11:19:07.900293 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '07f504cc-a04d-4738-8be7-ff333195f9d9', 'tag': '11c2ccd2-652a-4862-8c69-587b35b23237'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:19:07.900415 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1ff26346-8675-43df-b264-74dc1c913b76 None None] Pushing event updated for resources: {'Port': ['ID=5646b226-211d-4c06-b7f8-5dfc132e817e,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:07.920176 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-1ff26346-8675-43df-b264-74dc1c913b76 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 1.170s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:07.936584 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f47c3763-8516-4735-8160-2d19a43c31e4 req-f12dde3a-f992-4daa-a10f-7c95060416b0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1548700 May 24 11:19:07.944830 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Pushing event updated for resources: {'Port': ['ID=11c2ccd2-652a-4862-8c69-587b35b23237,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:07.963232 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.158s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:08.053333 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2df3e20d-8580-4abb-b562-ae817939b96f req-44176bcc-38e9-4284-94d6-162cfc56f859 service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3270283 May 24 11:19:08.055905 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Network e6b6c00d-d044-4bc7-92ce-02e445f1f072 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91820) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:19:08.079110 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, e6b6c00d-d044-4bc7-92ce-02e445f1f072) {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:19:08.098952 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:08.102281 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:08.105807 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-9b563f9f-f2fe-4853-9b01-5bfb97d12247 None None] Network e6b6c00d-d044-4bc7-92ce-02e445f1f072 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:19:08.202361 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-d3f7722e-5dfc-4ce1-8445-8782a61fce8c admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 848 time: 2.5897276 May 24 11:19:08.241265 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 56468) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:08.253144 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Request body: {'router': {'name': 'tempest-TestVolumeBootPattern-1369558850-router', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:08.262437 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:08.266844 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:19:08.267587 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Usage tracker for resource:router and tenant:672c5f4985084a8492a86667aceb0768 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:08.274111 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:08.298959 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:08.326056 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-036be8a5-ef04-421c-8912-9e0a19face80 req-ab53f847-123c-40ea-802e-2e1d80ad7b49 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:08.328505 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-ab53f847-123c-40ea-802e-2e1d80ad7b49 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=7c9e587a510c4777ab57643ed31ea73f&device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 1097 time: 0.2882853 May 24 11:19:08.344871 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-684314', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4157'] for router, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:08.346337 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-684570', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-687045', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-715560', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4164'] for router, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:08.382185 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f758c3ba-33f2-4d8e-b43b-0fcb97d5236e req-728bffe7-98c8-43e1-b7e7-da709e05b84b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:08.383806 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f758c3ba-33f2-4d8e-b43b-0fcb97d5236e req-728bffe7-98c8-43e1-b7e7-da709e05b84b service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 1099 time: 0.2522318 May 24 11:19:08.448109 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d4605809-f7dc-44f6-9e78-5bf4b3fd0c74 req-bb21728c-7ec0-4b0a-b106-2c9a4b8aba70 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:08.450244 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d4605809-f7dc-44f6-9e78-5bf4b3fd0c74 req-bb21728c-7ec0-4b0a-b106-2c9a4b8aba70 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 944 time: 0.3201637 May 24 11:19:08.456380 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:08.521426 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f758c3ba-33f2-4d8e-b43b-0fcb97d5236e req-068ef6ef-a3b0-4ac9-a78a-ac4aa5baa1ee service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1299143 May 24 11:19:08.536792 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d4605809-f7dc-44f6-9e78-5bf4b3fd0c74 req-7e7d4d23-05d6-4b0b-a0e8-0850085b8f7c tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0735497 May 24 11:19:08.690263 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '07f504cc-a04d-4738-8be7-ff333195f9d9', 'tag': '11c2ccd2-652a-4862-8c69-587b35b23237', 'status': 'completed', 'code': 200} May 24 11:19:08.692588 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-f8cf7ccb-6c0c-4838-9f27-648a847373d0 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=11c2ccd2-652a-4862-8c69-587b35b23237 HTTP/1.1" status: 200 len: 217 time: 0.3559747 May 24 11:19:08.715937 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ffaee36a-0515-49de-b402-43910d30b4ca req-32b305a9-62bc-4ac2-837e-539076199c1e tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:08.727733 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ffaee36a-0515-49de-b402-43910d30b4ca req-32b305a9-62bc-4ac2-837e-539076199c1e tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.5477602 May 24 11:19:08.843881 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ffaee36a-0515-49de-b402-43910d30b4ca req-bbe2887b-720e-4eda-9c9c-ff6b16505c95 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1149719 May 24 11:19:08.888489 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-036be8a5-ef04-421c-8912-9e0a19face80 req-02d58d9c-5cc4-4bd5-83b7-80ebdb87d202 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:08.892049 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-02d58d9c-5cc4-4bd5-83b7-80ebdb87d202 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=1ed78d41-8564-48a2-acc1-2ef21df88d9e HTTP/1.1" status: 200 len: 849 time: 0.1334577 May 24 11:19:09.022514 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-092ff1e6-9628-4097-8b9e-987669afaf88 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=a21e7e1e-54f7-4844-b98d-f658bdb26a4c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1224811 May 24 11:19:09.084306 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpea482e27-f15a-5756-a111-7e0dba035002-e6b6c00d-d044-4bc7-92ce-02e445f1f072', 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'fixed_ips': [{'subnet_id': 'a25edff7-c0c6-44d4-ad1c-a38e28a26a35'}]}} from dv-cinder-784092-9-iscsi. {{(pid=91821) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 11:19:09.085901 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:09.259257 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-3d2d3da4-854e-4e36-b69d-c18690062142 service neutron] 10.222.0.93 "GET /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2280858 May 24 11:19:09.282562 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-687072', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-684328', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4178'] for router, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:09.451430 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:06Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:09.501503 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3cf2d3a0-ada3-4ad2-8c55-3e6ece3a8cdb None None] The sync data for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi is: [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:06Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:19:09.526779 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.db_base_plugin_common [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Allocated IP 172.20.0.2 (e6b6c00d-d044-4bc7-92ce-02e445f1f072/a25edff7-c0c6-44d4-ad1c-a38e28a26a35/74b6164f-9aab-4c25-bfee-4839f99ffe91) {{(pid=91821) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:19:09.648596 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7930b09-424e-47db-9455-417c5a002ef6 req-545ec721-0cd3-4e10-b85f-2b7bb98fe8c2 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:09.656416 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a7930b09-424e-47db-9455-417c5a002ef6 req-545ec721-0cd3-4e10-b85f-2b7bb98fe8c2 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.4389324 May 24 11:19:09.774914 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-036be8a5-ef04-421c-8912-9e0a19face80 req-d2957fa5-4672-415c-a070-859d8528f5df service neutron] 10.222.0.93 "GET /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.5010223 May 24 11:19:09.805006 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a7930b09-424e-47db-9455-417c5a002ef6 req-c17ca012-2883-49f4-9c53-148f6dec1794 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1337521 May 24 11:19:10.320023 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:10.358813 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e5e02ed-c0d6-4c98-9b73-798e8ee0508e req-8b5cdac1-e4ea-45fa-ae8d-510831c9a3a1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:10.361462 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6e5e02ed-c0d6-4c98-9b73-798e8ee0508e req-8b5cdac1-e4ea-45fa-ae8d-510831c9a3a1 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=7c9e587a510c4777ab57643ed31ea73f&device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 1097 time: 0.3083549 May 24 11:19:10.364225 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:10.365285 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Notify callbacks [] for port, precommit_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:10.394545 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Transition to ACTIVE for port object 74b6164f-9aab-4c25-bfee-4839f99ffe91 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:19:10.403231 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca1ca5ad-3fc9-4261-af49-71d92c15437e req-4cc00735-bec6-4786-8dc4-22dcf9aed79a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:10.418908 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ca1ca5ad-3fc9-4261-af49-71d92c15437e req-4cc00735-bec6-4786-8dc4-22dcf9aed79a tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 944 time: 0.5676525 May 24 11:19:10.474390 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-296bc222-8ea7-4504-b5e0-da0441856b14 req-08bdc1f1-f16e-48ce-8388-55a4a265f3dc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:10.476405 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-296bc222-8ea7-4504-b5e0-da0441856b14 req-08bdc1f1-f16e-48ce-8388-55a4a265f3dc tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.3743279 May 24 11:19:10.476663 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Transition to ACTIVE for port object 74b6164f-9aab-4c25-bfee-4839f99ffe91 will not be triggered until provisioned by entity DHCP. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:19:10.493150 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:10.511368 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-d0235251-adeb-4c11-887e-5fc4021ddf68 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:10.592712 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 5646b226-211d-4c06-b7f8-5dfc132e817e up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:10.664762 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ca1ca5ad-3fc9-4261-af49-71d92c15437e req-9e890f84-f571-479a-b191-faa174c72380 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2445736 May 24 11:19:10.725498 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e5e02ed-c0d6-4c98-9b73-798e8ee0508e req-1988bd38-5ef8-4040-a8d4-fdba5f0dd86c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:10.727992 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6e5e02ed-c0d6-4c98-9b73-798e8ee0508e req-1988bd38-5ef8-4040-a8d4-fdba5f0dd86c service neutron] 10.222.0.93 "GET /v2.0/networks?id=a21e7e1e-54f7-4844-b98d-f658bdb26a4c HTTP/1.1" status: 200 len: 910 time: 0.2403438 May 24 11:19:10.747692 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-296bc222-8ea7-4504-b5e0-da0441856b14 req-6723ab52-0450-43ce-9f64-46cb26248e3b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2570040 May 24 11:19:10.766807 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Allocated IP 172.27.0.205 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/a4cc0b23-ca4a-4a18-9cee-ce596875baf8) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:19:10.818719 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Network e6b6c00d-d044-4bc7-92ce-02e445f1f072 is already hosted by enough agents. {{(pid=91821) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:19:10.824397 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Pushing event updated for resources: {'Port': ['ID=74b6164f-9aab-4c25-bfee-4839f99ffe91,revision_number=1']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:10.829953 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.base_scheduler [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Resources bound (agent type, host, resource id): {{(pid=91821) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:19:10.845509 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.335s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:10.846304 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Received port after_create {{(pid=91821) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:19:10.929365 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6e5e02ed-c0d6-4c98-9b73-798e8ee0508e req-47ac8c8b-40be-4aa6-81a1-8f73de21a5c7 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=11c2ccd2-652a-4862-8c69-587b35b23237 HTTP/1.1" status: 200 len: 217 time: 0.1879985 May 24 11:19:11.082377 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e5e02ed-c0d6-4c98-9b73-798e8ee0508e req-c9f2db4d-6705-45dc-a95a-0d16de4166aa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:11.086180 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 5646b226-211d-4c06-b7f8-5dfc132e817e completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:19:11.087477 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6e5e02ed-c0d6-4c98-9b73-798e8ee0508e req-c9f2db4d-6705-45dc-a95a-0d16de4166aa service neutron] 10.222.0.93 "GET /v2.0/subnets?id=1ed78d41-8564-48a2-acc1-2ef21df88d9e HTTP/1.1" status: 200 len: 849 time: 0.1438901 May 24 11:19:11.096748 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:11.108597 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 5646b226-211d-4c06-b7f8-5dfc132e817e triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:11.109242 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:11.188806 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:11.196547 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:11.228683 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Attempting to bind port 74b6164f-9aab-4c25-bfee-4839f99ffe91 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:19:11.229219 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Attempting to bind port 74b6164f-9aab-4c25-bfee-4839f99ffe91 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'c1fbdc79-842c-49eb-8789-2a1de3ab2b9f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 520, 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:19:11.229490 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Attempting to bind port 74b6164f-9aab-4c25-bfee-4839f99ffe91 on network e6b6c00d-d044-4bc7-92ce-02e445f1f072 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:19:11.234568 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6e5e02ed-c0d6-4c98-9b73-798e8ee0508e req-ed6ba256-3194-49ee-974c-e7e46f9229bf service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=a21e7e1e-54f7-4844-b98d-f658bdb26a4c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1255753 May 24 11:19:11.280350 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:19:11.280733 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Checking segment: {'id': 'c1fbdc79-842c-49eb-8789-2a1de3ab2b9f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 520, 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:19:11.281174 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:11.281588 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Bound using segment: {'id': 'c1fbdc79-842c-49eb-8789-2a1de3ab2b9f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 520, 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:19:11.282084 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Bound port: 74b6164f-9aab-4c25-bfee-4839f99ffe91, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'c1fbdc79-842c-49eb-8789-2a1de3ab2b9f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 520, 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:19:11.282463 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:11.419552 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:19:11.422985 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:19:11.440914 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:19:11.501247 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Pushing event updated for resources: {'Port': ['ID=a4cc0b23-ca4a-4a18-9cee-ce596875baf8,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:11.510707 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.314s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:11.633156 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6e5e02ed-c0d6-4c98-9b73-798e8ee0508e req-7568941a-cbb2-43f9-a86b-0182cb8b3f4f service neutron] 10.222.0.93 "GET /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3905053 May 24 11:19:11.651637 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] For port 74b6164f-9aab-4c25-bfee-4839f99ffe91, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:19:11.656414 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] For port 74b6164f-9aab-4c25-bfee-4839f99ffe91, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=74b6164f-9aab-4c25-bfee-4839f99ffe91,segment=NetworkSegment(c1fbdc79-842c-49eb-8789-2a1de3ab2b9f),segment_id=c1fbdc79-842c-49eb-8789-2a1de3ab2b9f)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:19:11.659172 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:11.674263 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:11.682398 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-8b80e5e1-c716-4acc-a7ad-4e3268b74df1 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:11.783461 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 5646b226-211d-4c06-b7f8-5dfc132e817e, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=5646b226-211d-4c06-b7f8-5dfc132e817e,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:11.800222 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:11.811792 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Pushing event updated for resources: {'Port': ['ID=74b6164f-9aab-4c25-bfee-4839f99ffe91,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:11.815217 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.133s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:11.816235 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.013s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:11.984295 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=5646b226-211d-4c06-b7f8-5dfc132e817e,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:11.991113 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.175s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:12.158368 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6e5e02ed-c0d6-4c98-9b73-798e8ee0508e req-cf3a15c3-90c4-47c2-881f-6b38a2ab7b55 service neutron] 10.222.0.93 "GET /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.5051055 May 24 11:19:12.198304 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e317468-2b13-43f4-bc6d-fbf4b4a7b4a7 req-97061fc3-cec5-49cb-9201-a9495e64c762 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:12.200832 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4e317468-2b13-43f4-bc6d-fbf4b4a7b4a7 req-97061fc3-cec5-49cb-9201-a9495e64c762 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 944 time: 0.2682414 May 24 11:19:12.298967 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4e317468-2b13-43f4-bc6d-fbf4b4a7b4a7 req-1e618a37-0ae9-4dc8-a5df-7f3cc79cfb87 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0825629 May 24 11:19:12.408364 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-adfaaf47-8efe-4b8a-8eb5-92535d80693b req-13d9912d-245f-4beb-b140-b19e763e0f23 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:12.412350 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-adfaaf47-8efe-4b8a-8eb5-92535d80693b req-13d9912d-245f-4beb-b140-b19e763e0f23 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 1099 time: 0.5835333 May 24 11:19:12.417990 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:12.524531 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1293760-9674-483b-9a2b-96048be15a1a req-fe1b5333-38b8-489c-9770-6e68ee89c6c4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:12.526094 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d1293760-9674-483b-9a2b-96048be15a1a req-fe1b5333-38b8-489c-9770-6e68ee89c6c4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.4449961 May 24 11:19:12.553568 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 74b6164f-9aab-4c25-bfee-4839f99ffe91 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:12.695669 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 74b6164f-9aab-4c25-bfee-4839f99ffe91 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:19:12.708985 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 5646b226-211d-4c06-b7f8-5dfc132e817e up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:12.810816 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 5646b226-211d-4c06-b7f8-5dfc132e817e triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:12.811273 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:12.885874 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d1293760-9674-483b-9a2b-96048be15a1a req-d6191510-554a-4595-bb83-080cf2231454 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1122231 May 24 11:19:12.988571 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-adfaaf47-8efe-4b8a-8eb5-92535d80693b req-7a3491f2-cc95-4389-afe7-73779be0db68 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:12.990506 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-adfaaf47-8efe-4b8a-8eb5-92535d80693b req-7a3491f2-cc95-4389-afe7-73779be0db68 service neutron] 10.222.0.93 "GET /v2.0/networks?id=4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 200 len: 918 time: 0.4991376 May 24 11:19:13.044077 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 11c2ccd2-652a-4862-8c69-587b35b23237 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:19:13.262748 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 11c2ccd2-652a-4862-8c69-587b35b23237, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=11c2ccd2-652a-4862-8c69-587b35b23237,segment=NetworkSegment(f3295bfe-6894-409e-b0cc-96311bd90247),segment_id=f3295bfe-6894-409e-b0cc-96311bd90247)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:13.307631 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '11c2ccd2-652a-4862-8c69-587b35b23237', 'network_id': 'a21e7e1e-54f7-4844-b98d-f658bdb26a4c', 'port_id': '11c2ccd2-652a-4862-8c69-587b35b23237', 'mac_address': 'fa:16:3e:8c:f4:56', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 514, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '1ed78d41-8564-48a2-acc1-2ef21df88d9e', 'ip_address': '172.20.0.10'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:19:13.339742 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-adfaaf47-8efe-4b8a-8eb5-92535d80693b req-660993a8-2a1f-497a-8f88-19fcaf9ad8fd service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=02210ade-8c60-41b6-aa82-a104d4fc0447 HTTP/1.1" status: 200 len: 217 time: 0.3357189 May 24 11:19:13.447308 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-57982877-c8d3-494b-b021-349b7c31cef5 None None] Network e6b6c00d-d044-4bc7-92ce-02e445f1f072 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:19:13.469913 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-adfaaf47-8efe-4b8a-8eb5-92535d80693b req-b52095b2-f9dd-46b9-b547-d913df436a07 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:13.471368 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-adfaaf47-8efe-4b8a-8eb5-92535d80693b req-b52095b2-f9dd-46b9-b547-d913df436a07 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=52267672-fd8c-45bd-89ea-d9d02a7b249a HTTP/1.1" status: 200 len: 857 time: 0.1266997 May 24 11:19:13.590064 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-adfaaf47-8efe-4b8a-8eb5-92535d80693b req-69420424-0135-4bef-8d46-d78e8da2e8bf service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=4fec6b68-64f1-4553-8ca5-f86c63ac9714&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1107178 May 24 11:19:13.647707 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:13.689641 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 11c2ccd2-652a-4862-8c69-587b35b23237 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:19:13.702387 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-e3a7fb51-6815-432a-aa85-d77e1a9c8339 None None] Provisioning for port 74b6164f-9aab-4c25-bfee-4839f99ffe91 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:19:13.720258 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-e3a7fb51-6815-432a-aa85-d77e1a9c8339 None None] Provisioning complete for port 74b6164f-9aab-4c25-bfee-4839f99ffe91 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:13.720648 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e3a7fb51-6815-432a-aa85-d77e1a9c8339 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:13.796713 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad25038e-7bae-4907-ac79-7be7fd9132f9 req-dfc7067b-41b4-4393-a4ba-de9e9cf19aea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:13.798444 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ad25038e-7bae-4907-ac79-7be7fd9132f9 req-dfc7067b-41b4-4393-a4ba-de9e9cf19aea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 944 time: 0.3237615 May 24 11:19:13.799449 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-684307', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-687011'] for router, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:13.851210 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 11c2ccd2-652a-4862-8c69-587b35b23237, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=11c2ccd2-652a-4862-8c69-587b35b23237,segment=NetworkSegment(f3295bfe-6894-409e-b0cc-96311bd90247),segment_id=f3295bfe-6894-409e-b0cc-96311bd90247)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:13.859519 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:13.867800 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:14.001993 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=11c2ccd2-652a-4862-8c69-587b35b23237,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:14.008097 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.140s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:14.020337 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e3a7fb51-6815-432a-aa85-d77e1a9c8339 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:14.040215 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-adfaaf47-8efe-4b8a-8eb5-92535d80693b req-e02b238f-7464-4e6d-9a6c-8fb850149bbf service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4352272 May 24 11:19:14.056556 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ad25038e-7bae-4907-ac79-7be7fd9132f9 req-2c78c79a-2e59-4bb1-befa-2d8fedf8bb36 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1664035 May 24 11:19:14.100747 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-e3a7fb51-6815-432a-aa85-d77e1a9c8339 None None] For port 74b6164f-9aab-4c25-bfee-4839f99ffe91, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=74b6164f-9aab-4c25-bfee-4839f99ffe91,segment=NetworkSegment(c1fbdc79-842c-49eb-8789-2a1de3ab2b9f),segment_id=c1fbdc79-842c-49eb-8789-2a1de3ab2b9f)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:14.110070 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e3a7fb51-6815-432a-aa85-d77e1a9c8339 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:14.112297 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:14.217948 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e3a7fb51-6815-432a-aa85-d77e1a9c8339 None None] Pushing event updated for resources: {'Port': ['ID=74b6164f-9aab-4c25-bfee-4839f99ffe91,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:14.230075 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-e3a7fb51-6815-432a-aa85-d77e1a9c8339 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.118s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:14.240817 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['11c2ccd2-652a-4862-8c69-587b35b23237'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:19:14.249511 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-047b791d-0de7-4da8-aa1e-b200afaa024a req-59fd2bdb-c093-4f94-8023-bc7a63ac5e38 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:14.257584 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-047b791d-0de7-4da8-aa1e-b200afaa024a req-59fd2bdb-c093-4f94-8023-bc7a63ac5e38 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1495461 May 24 11:19:14.381764 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-047b791d-0de7-4da8-aa1e-b200afaa024a req-700dd885-ad62-40ac-9a8b-ba019d8c5f98 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1229284 May 24 11:19:14.411536 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-adfaaf47-8efe-4b8a-8eb5-92535d80693b req-a02a1820-6199-4d83-9280-816ef128457c service neutron] 10.222.0.93 "GET /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3544974 May 24 11:19:14.451139 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 74b6164f-9aab-4c25-bfee-4839f99ffe91 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:14.476119 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Router 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e is scheduled to L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 with binding_index 1 {{(pid=91820) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 11:19:14.579415 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-5d14cb00-e70f-4689-9bcf-2bba6b79e89b None None] Provisioning complete for port 74b6164f-9aab-4c25-bfee-4839f99ffe91 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:14.579794 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-5d14cb00-e70f-4689-9bcf-2bba6b79e89b None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:14.663836 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 74b6164f-9aab-4c25-bfee-4839f99ffe91 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:14.664191 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:15.148700 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:19:15.159188 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Sync routers for ids ['7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:19:15.186281 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Routers {'7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:19:15.187714 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:15.198156 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Notify callbacks [] for router, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:15.201142 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] 10.222.0.18 "POST /v2.0/routers HTTP/1.1" status: 201 len: 924 time: 6.9523840 May 24 11:19:15.202654 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 56526) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:15.424577 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Set router 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:19:15.536542 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d4a02f7-fc9f-40b3-9e6b-a4cc600851d0 req-f0fcf57b-c8b2-4201-82b5-28c8d343be19 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:15.538324 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1d4a02f7-fc9f-40b3-9e6b-a4cc600851d0 req-f0fcf57b-c8b2-4201-82b5-28c8d343be19 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 945 time: 0.2573881 May 24 11:19:15.644888 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a5c09698-c889-4f92-be5f-fae14e7aaa50 req-30b41e6d-b41c-4cf5-bb5f-be0477bcb9ff tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:15.653635 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a5c09698-c889-4f92-be5f-fae14e7aaa50 req-30b41e6d-b41c-4cf5-bb5f-be0477bcb9ff tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1190088 May 24 11:19:15.724385 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1d4a02f7-fc9f-40b3-9e6b-a4cc600851d0 req-bf3752bd-5f83-4f80-bf1a-71b069738a38 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0838947 May 24 11:19:15.760391 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a5c09698-c889-4f92-be5f-fae14e7aaa50 req-a20d7b68-e0af-4853-9c6d-1a222d976b4f tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1048613 May 24 11:19:15.873026 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'name': 'tempest-TestVolumeBootPattern-1369558850-router', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205'}], 'enable_snat': True}, 'gw_port_id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:19:08Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 3, 'project_id': '672c5f4985084a8492a86667aceb0768', 'gw_port': {'id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:68:b5:40', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 403, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:19:10Z', 'updated_at': '2021-05-24T11:19:10Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:15.877897 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Checking router: 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:19:15.878562 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Updating router 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e port a4cc0b23-ca4a-4a18-9cee-ce596875baf8 binding host dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:19:15.954725 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:16.162369 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:16.432700 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Allocated IP 172.20.0.1 (e6b6c00d-d044-4bc7-92ce-02e445f1f072/a25edff7-c0c6-44d4-ad1c-a38e28a26a35/cbcf156d-db06-40df-9ad0-c2981b742561) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:19:16.488517 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Notify callbacks [] for port, precommit_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:16.516994 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.provisioning_blocks [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Transition to ACTIVE for port object a4cc0b23-ca4a-4a18-9cee-ce596875baf8 will not be triggered until provisioned by entity L2. {{(pid=91822) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:19:16.539701 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:16.569522 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-1f6f5c39-61be-4156-8d1a-d54d91752876 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:16.702275 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:16.709839 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Attempting to bind port a4cc0b23-ca4a-4a18-9cee-ce596875baf8 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:19:16.710646 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Attempting to bind port a4cc0b23-ca4a-4a18-9cee-ce596875baf8 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:19:16.711265 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Attempting to bind port a4cc0b23-ca4a-4a18-9cee-ce596875baf8 on network 858b8fad-a73f-4578-9079-6e731b9ff7e2 {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:19:16.725645 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Transition to ACTIVE for port object cbcf156d-db06-40df-9ad0-c2981b742561 will not be triggered until provisioned by entity DHCP. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:19:16.730965 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-96e21e33-f197-465c-8afc-10484f574a70 None None] Pushing event updated for resources: {'Port': ['ID=a4cc0b23-ca4a-4a18-9cee-ce596875baf8,revision_number=2']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:16.736031 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-96e21e33-f197-465c-8afc-10484f574a70 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.167s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:16.737148 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:19:16.737669 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Checking segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91822) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:19:16.738176 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:16.738633 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Bound using segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:19:16.739179 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Bound port: a4cc0b23-ca4a-4a18-9cee-ce596875baf8, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:19:16.739758 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:16.791403 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:16.797329 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:16.871283 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Network e6b6c00d-d044-4bc7-92ce-02e445f1f072 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:19:16.875978 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:19:16.882572 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:19:16.911827 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] For port a4cc0b23-ca4a-4a18-9cee-ce596875baf8, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91822) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:19:16.921850 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] For port a4cc0b23-ca4a-4a18-9cee-ce596875baf8, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=a4cc0b23-ca4a-4a18-9cee-ce596875baf8,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91822) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:19:16.931387 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:16.946212 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:16.952359 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-687570'] for router_interface, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:17.020790 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-11188b13-28f0-49b2-a5d6-deb05bbff785 req-ec326977-6066-44eb-80c8-26c9a207f19d tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:17.022427 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-11188b13-28f0-49b2-a5d6-deb05bbff785 req-ec326977-6066-44eb-80c8-26c9a207f19d tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 946 time: 0.1928675 May 24 11:19:17.031452 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Pushing event updated for resources: {'Port': ['ID=cbcf156d-db06-40df-9ad0-c2981b742561,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:17.035008 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Routers {'7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:19:17.035792 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:17.058878 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.249s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:17.086965 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 11c2ccd2-652a-4862-8c69-587b35b23237 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:17.126123 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-11188b13-28f0-49b2-a5d6-deb05bbff785 req-2ed15345-f16f-418c-9f79-5711cea90d73 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0791965 May 24 11:19:17.167266 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5cae692-2233-4c5d-8463-542c3d2cfc4e req-5758e64e-27ec-4241-a4c1-7109e4a266eb tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:17.168908 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e5cae692-2233-4c5d-8463-542c3d2cfc4e req-5758e64e-27ec-4241-a4c1-7109e4a266eb tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 945 time: 0.2105818 May 24 11:19:17.289488 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-96e21e33-f197-465c-8afc-10484f574a70 None None] Pushing event updated for resources: {'Port': ['ID=a4cc0b23-ca4a-4a18-9cee-ce596875baf8,revision_number=3']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:17.295019 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-96e21e33-f197-465c-8afc-10484f574a70 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.349s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:17.320812 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 11c2ccd2-652a-4862-8c69-587b35b23237 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:19:17.329802 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 11c2ccd2-652a-4862-8c69-587b35b23237 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:17.330182 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:17.378094 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] Set router 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:19:17.455199 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-7247f55d-64bd-4c2e-8334-4dc9505564bb None None] Provisioning for port cbcf156d-db06-40df-9ad0-c2981b742561 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:19:17.470892 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-7247f55d-64bd-4c2e-8334-4dc9505564bb None None] Provisioning complete for port cbcf156d-db06-40df-9ad0-c2981b742561 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:17.471282 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-7247f55d-64bd-4c2e-8334-4dc9505564bb None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:17.505358 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:17.541680 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce97694d-d9f4-4c16-a558-951d4f3fa053 req-4db0f55d-4731-4f90-9101-fd5d02bd21ab tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:17.549958 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ce97694d-d9f4-4c16-a558-951d4f3fa053 req-4db0f55d-4731-4f90-9101-fd5d02bd21ab tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.2128780 May 24 11:19:17.599261 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e5cae692-2233-4c5d-8463-542c3d2cfc4e req-94e943cf-6ca2-40e2-85b1-64ecb3ebca31 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.4232450 May 24 11:19:17.640011 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-7247f55d-64bd-4c2e-8334-4dc9505564bb None None] Port cbcf156d-db06-40df-9ad0-c2981b742561 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:19:17.640566 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f023b39b-1add-48c8-920b-532b6c02c2a9 req-493a7733-12a5-4610-84f0-9dbe744d7d71 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1843083 May 24 11:19:17.669173 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:17.686887 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ce97694d-d9f4-4c16-a558-951d4f3fa053 req-ec1c5aa9-e307-460b-a17a-1cc7a3ef4937 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1366775 May 24 11:19:17.716982 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 11c2ccd2-652a-4862-8c69-587b35b23237, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=11c2ccd2-652a-4862-8c69-587b35b23237,segment=NetworkSegment(f3295bfe-6894-409e-b0cc-96311bd90247),segment_id=f3295bfe-6894-409e-b0cc-96311bd90247)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:17.718151 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '07f504cc-a04d-4738-8be7-ff333195f9d9', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '11c2ccd2-652a-4862-8c69-587b35b23237'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:19:17.726554 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:17.728635 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:17.807319 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '07f504cc-a04d-4738-8be7-ff333195f9d9', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '11c2ccd2-652a-4862-8c69-587b35b23237', 'code': 200} May 24 11:19:17.849697 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=11c2ccd2-652a-4862-8c69-587b35b23237,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:17.865552 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.137s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:18.080982 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f023b39b-1add-48c8-920b-532b6c02c2a9 req-25e9e27a-7b51-482e-a9d4-2befd9999aea service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:18.083249 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f023b39b-1add-48c8-920b-532b6c02c2a9 req-25e9e27a-7b51-482e-a9d4-2befd9999aea service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=b303f83a84794e5ba6a1bab9c76b429c&device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 1099 time: 0.3993151 May 24 11:19:18.125757 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'name': 'tempest-TestVolumeBootPattern-1369558850-router', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205'}], 'enable_snat': True}, 'gw_port_id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:19:08Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 3, 'project_id': '672c5f4985084a8492a86667aceb0768', 'gw_port': {'id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:68:b5:40', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 403, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:19:10Z', 'updated_at': '2021-05-24T11:19:16Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:18.135697 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2ddbd793-313c-430f-a26f-28809208f356 None None] The sync data for ids ['7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'] in dv-cinder-784092-9-iscsi is: [{'id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'name': 'tempest-TestVolumeBootPattern-1369558850-router', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205'}], 'enable_snat': True}, 'gw_port_id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:19:08Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 3, 'project_id': '672c5f4985084a8492a86667aceb0768', 'gw_port': {'id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:68:b5:40', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 403, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:19:10Z', 'updated_at': '2021-05-24T11:19:16Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:19:18.166678 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:18.170441 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0f613c8a-b9ff-44f2-a393-785e9fcf9b26 req-f977d749-fcfd-4fa9-8451-214cd52c1282 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:18.171930 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0f613c8a-b9ff-44f2-a393-785e9fcf9b26 req-f977d749-fcfd-4fa9-8451-214cd52c1282 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 1099 time: 0.1317794 May 24 11:19:18.298077 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0f613c8a-b9ff-44f2-a393-785e9fcf9b26 req-8403c528-2440-4b5e-bfbe-1ead596a1488 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1177835 May 24 11:19:18.327171 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:18.398827 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-57a43a3b-c8fe-49da-8a3b-7df2c8fab90e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:18.544383 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-754d958e-814f-4ff9-92a8-5696ca07f1d9 req-57f74a79-7d2c-4a0f-ad63-9d283143448e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:18.545938 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-754d958e-814f-4ff9-92a8-5696ca07f1d9 req-57f74a79-7d2c-4a0f-ad63-9d283143448e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 946 time: 0.1222420 May 24 11:19:18.647582 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-533332', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-687487'] for router_interface, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:18.674577 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Pushing event updated for resources: {'Port': ['ID=cbcf156d-db06-40df-9ad0-c2981b742561,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:18.688075 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.289s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:18.783057 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f023b39b-1add-48c8-920b-532b6c02c2a9 req-329883f2-3df3-4b9b-ac2e-188ac38e0a04 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:18.787205 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f023b39b-1add-48c8-920b-532b6c02c2a9 req-329883f2-3df3-4b9b-ac2e-188ac38e0a04 service neutron] 10.222.0.93 "GET /v2.0/networks?id=1d92641a-d8f9-4ed6-9abd-f464bb26811d HTTP/1.1" status: 200 len: 912 time: 0.6742508 May 24 11:19:18.902416 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-754d958e-814f-4ff9-92a8-5696ca07f1d9 req-521dd897-ab3b-4ff9-89b0-e0650b8dcba5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Deleting port bb147152-8c96-4e6d-b982-2e6076cc6145 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:19:18.902856 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-754d958e-814f-4ff9-92a8-5696ca07f1d9 req-521dd897-ab3b-4ff9-89b0-e0650b8dcba5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:18.932473 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2d701d5-fbc3-4922-85a5-f21d85d811eb req-2c7a3759-72e7-42c3-88fe-dbfb5d4adc8c tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:18.941195 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d2d701d5-fbc3-4922-85a5-f21d85d811eb req-2c7a3759-72e7-42c3-88fe-dbfb5d4adc8c tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 946 time: 0.5452917 May 24 11:19:19.001923 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68de1e68-2e62-4882-947f-7658c6d27591 req-2c8b856c-27f7-487f-bc5a-5e1915922ab5 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:19.004109 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-68de1e68-2e62-4882-947f-7658c6d27591 req-2c8b856c-27f7-487f-bc5a-5e1915922ab5 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 946 time: 0.2289398 May 24 11:19:19.086214 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f023b39b-1add-48c8-920b-532b6c02c2a9 req-cc3350f1-7a03-4db4-8034-9443927e9c14 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=eab44150-44f7-4194-9b76-436d421794e8 HTTP/1.1" status: 200 len: 1001 time: 0.2906132 May 24 11:19:19.108468 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-68de1e68-2e62-4882-947f-7658c6d27591 req-37fb025c-270c-431e-bf51-ca5d3acae472 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0916028 May 24 11:19:19.151775 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-754d958e-814f-4ff9-92a8-5696ca07f1d9 req-521dd897-ab3b-4ff9-89b0-e0650b8dcba5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] For port bb147152-8c96-4e6d-b982-2e6076cc6145, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=bb147152-8c96-4e6d-b982-2e6076cc6145,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:19.152196 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-754d958e-814f-4ff9-92a8-5696ca07f1d9 req-521dd897-ab3b-4ff9-89b0-e0650b8dcba5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:19.183939 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d2d701d5-fbc3-4922-85a5-f21d85d811eb req-6b8ad72b-97d3-4a72-8c0b-e925fbc3b6fc tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2417645 May 24 11:19:19.185990 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Network e6b6c00d-d044-4bc7-92ce-02e445f1f072 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:19:19.194136 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:19:19.379368 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-754d958e-814f-4ff9-92a8-5696ca07f1d9 req-521dd897-ab3b-4ff9-89b0-e0650b8dcba5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Calling delete_port for bb147152-8c96-4e6d-b982-2e6076cc6145 owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:19:19.618480 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Router 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:19:19.625609 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-754d958e-814f-4ff9-92a8-5696ca07f1d9 req-521dd897-ab3b-4ff9-89b0-e0650b8dcba5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:19.642129 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['45911d6f-9595-40c5-8b29-75259c824bfe', '159fe3b9-426c-472e-9870-2ead1792535a', '516f8f18-b5b5-4ded-bdfd-c420f5651d69'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:19:19.652111 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f023b39b-1add-48c8-920b-532b6c02c2a9 req-85577993-b889-4130-9faa-6fe67674d253 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:19.654071 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f023b39b-1add-48c8-920b-532b6c02c2a9 req-85577993-b889-4130-9faa-6fe67674d253 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=2fc4be2e-875c-4d10-b857-a48f90c53cf2 HTTP/1.1" status: 200 len: 850 time: 0.5261166 May 24 11:19:19.655366 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:19.777251 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-521dd897-ab3b-4ff9-89b0-e0650b8dcba5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Pushing event deleted for resources: {'Port': ['ID=bb147152-8c96-4e6d-b982-2e6076cc6145,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:19.791864 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-521dd897-ab3b-4ff9-89b0-e0650b8dcba5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.137s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:19.815474 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-05186639-7c71-4d27-91ce-e5eda4fad01d None None] Provisioning complete for port cbcf156d-db06-40df-9ad0-c2981b742561 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:19.816954 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-05186639-7c71-4d27-91ce-e5eda4fad01d None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:19.917662 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-05186639-7c71-4d27-91ce-e5eda4fad01d None None] Port cbcf156d-db06-40df-9ad0-c2981b742561 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:19:20.001318 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-754d958e-814f-4ff9-92a8-5696ca07f1d9 req-521dd897-ab3b-4ff9-89b0-e0650b8dcba5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] No DVR routers for this DVR port bb147152-8c96-4e6d-b982-2e6076cc6145 on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:19:20.005985 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-754d958e-814f-4ff9-92a8-5696ca07f1d9 req-521dd897-ab3b-4ff9-89b0-e0650b8dcba5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:20.008848 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9233a97a-5cf6-4529-8a81-39073d46adae', 'name': 'network-vif-deleted', 'tag': 'bb147152-8c96-4e6d-b982-2e6076cc6145'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:19:20.060029 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-754d958e-814f-4ff9-92a8-5696ca07f1d9 req-521dd897-ab3b-4ff9-89b0-e0650b8dcba5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "DELETE /v2.0/ports/bb147152-8c96-4e6d-b982-2e6076cc6145 HTTP/1.1" status: 204 len: 173 time: 1.5009592 May 24 11:19:20.100218 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9233a97a-5cf6-4529-8a81-39073d46adae', 'name': 'network-vif-deleted', 'tag': 'bb147152-8c96-4e6d-b982-2e6076cc6145', 'status': 'completed', 'code': 200} May 24 11:19:20.589026 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-0402a001-f75c-49db-8a75-b6eb0971cbcc req-1a1370dc-b7d5-4c32-8303-d25aa968eb37 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Synchronizing usage tracker for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:20.592705 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device a4cc0b23-ca4a-4a18-9cee-ce596875baf8 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:20.627296 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-0402a001-f75c-49db-8a75-b6eb0971cbcc req-1a1370dc-b7d5-4c32-8303-d25aa968eb37 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Unset dirty status for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:20.629126 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0402a001-f75c-49db-8a75-b6eb0971cbcc req-1a1370dc-b7d5-4c32-8303-d25aa968eb37 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 210 time: 0.1573555 May 24 11:19:20.692634 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:19:20.712647 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f023b39b-1add-48c8-920b-532b6c02c2a9 req-1b815945-12c6-4649-80e5-5bdb431e1ba9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:20.714625 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f023b39b-1add-48c8-920b-532b6c02c2a9 req-1b815945-12c6-4649-80e5-5bdb431e1ba9 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=1d92641a-d8f9-4ed6-9abd-f464bb26811d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 1.0529785 May 24 11:19:20.721187 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] 10.222.0.18 "PUT /v2.0/routers/7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e/add_router_interface HTTP/1.1" status: 200 len: 503 time: 5.5171199 May 24 11:19:20.880280 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0c2240da-ebfe-48d7-b360-7b681f786c52 req-d7b9bc8c-5e6e-40aa-9b9c-73e1c0e0096d tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:20.888698 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0c2240da-ebfe-48d7-b360-7b681f786c52 req-d7b9bc8c-5e6e-40aa-9b9c-73e1c0e0096d tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 946 time: 0.4439244 May 24 11:19:20.980949 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port a4cc0b23-ca4a-4a18-9cee-ce596875baf8 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:19:21.020770 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port a4cc0b23-ca4a-4a18-9cee-ce596875baf8 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:21.021572 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:21.109371 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0c2240da-ebfe-48d7-b360-7b681f786c52 req-f81be203-66bc-4b7f-a9a4-d254ceb265e8 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2194901 May 24 11:19:21.118006 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f023b39b-1add-48c8-920b-532b6c02c2a9 req-51f4a179-13f9-4457-9741-244bdfaf10da service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3867784 May 24 11:19:21.120024 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Sync routers for ids ['7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:19:21.180835 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device bb147152-8c96-4e6d-b982-2e6076cc6145 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:19:21.443803 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: bb147152-8c96-4e6d-b982-2e6076cc6145 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:19:21.444265 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device bb147152-8c96-4e6d-b982-2e6076cc6145 not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:19:21.471502 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Routers {'7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:19:21.473928 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:21.548952 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f023b39b-1add-48c8-920b-532b6c02c2a9 req-f84dd2fd-83b8-4eb7-ac35-cd1cbb613725 service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4172707 May 24 11:19:21.828830 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:21.941966 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ae950e49-6231-43eb-a23b-c872489f29bc req-5df06c40-1fcd-4b6c-a50f-89c6dd4eb4d6 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=9233a97a-5cf6-4529-8a81-39073d46adae HTTP/1.1" status: 200 len: 210 time: 0.0806124 May 24 11:19:21.954690 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port a4cc0b23-ca4a-4a18-9cee-ce596875baf8, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=a4cc0b23-ca4a-4a18-9cee-ce596875baf8,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:21.992276 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:22.001844 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-e3a7fb51-6815-432a-aa85-d77e1a9c8339 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:22.102206 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:22.107709 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Set router 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:19:22.334568 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=a4cc0b23-ca4a-4a18-9cee-ce596875baf8,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:22.350588 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.349s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:22.547499 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'name': 'tempest-TestVolumeBootPattern-1369558850-router', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205'}], 'enable_snat': True}, 'gw_port_id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:19:08Z', 'updated_at': '2021-05-24T11:19:17Z', 'revision_number': 4, 'project_id': '672c5f4985084a8492a86667aceb0768', 'gw_port': {'id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:68:b5:40', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 403, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:19:10Z', 'updated_at': '2021-05-24T11:19:16Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'cbcf156d-db06-40df-9ad0-c2981b742561', 'name': '', 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'mac_address': 'fa:16:3e:82:35:f2', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'device_owner': 'network:router_interface', 'standard_attr_id': 404, 'fixed_ips': [{'subnet_id': 'a25edff7-c0c6-44d4-ad1c-a38e28a26a35', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:19:16Z', 'updated_at': '2021-05-24T11:19:17Z', 'revision_number': 2, 'project_id': '672c5f4985084a8492a86667aceb0768', 'subnets': [{'id': 'a25edff7-c0c6-44d4-ad1c-a38e28a26a35', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:22.552350 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Checking router: 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:19:22.552715 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Updating router 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e port cbcf156d-db06-40df-9ad0-c2981b742561 binding host dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:19:22.611793 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:22.984851 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 56626) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:22.991726 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-262467324', 'description': 'tempest-secgroup-smoke-262467324 description', 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:23.000250 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:23.006943 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Usage tracker for resource:security_group and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:23.007324 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Notify callbacks [] for port, precommit_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.011231 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:23.019383 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:23.022934 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Transition to ACTIVE for port object cbcf156d-db06-40df-9ad0-c2981b742561 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:19:23.030873 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.037355 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-597a084c-17d6-4f49-b1d0-a6d250776b93 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:23.037554 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18303bf9-7e11-43ef-918a-c6fc5681dacd req-93e6ecad-81fe-4118-aab0-b4732651816b tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:23.039152 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-18303bf9-7e11-43ef-918a-c6fc5681dacd req-93e6ecad-81fe-4118-aab0-b4732651816b tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 946 time: 0.1197224 May 24 11:19:23.068286 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.122002 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-18303bf9-7e11-43ef-918a-c6fc5681dacd req-cea32b15-6062-4767-adde-408c8d8d9b61 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0737128 May 24 11:19:23.123184 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:23.130984 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Usage tracker for resource:security_group_rule and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:23.134251 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:23.146111 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:23.236613 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d71c3a32-d104-46c1-8f8a-0c765c2e8c4a None None] Pushing event updated for resources: {'Port': ['ID=cbcf156d-db06-40df-9ad0-c2981b742561,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:23.251516 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-d71c3a32-d104-46c1-8f8a-0c765c2e8c4a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.214s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:23.330535 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Attempting to bind port cbcf156d-db06-40df-9ad0-c2981b742561 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:19:23.331287 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Attempting to bind port cbcf156d-db06-40df-9ad0-c2981b742561 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'c1fbdc79-842c-49eb-8789-2a1de3ab2b9f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 520, 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:19:23.331769 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Attempting to bind port cbcf156d-db06-40df-9ad0-c2981b742561 on network e6b6c00d-d044-4bc7-92ce-02e445f1f072 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:19:23.342037 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 18, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:19:23.342676 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Checking segment: {'id': 'c1fbdc79-842c-49eb-8789-2a1de3ab2b9f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 520, 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:19:23.343299 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.344392 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Bound using segment: {'id': 'c1fbdc79-842c-49eb-8789-2a1de3ab2b9f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 520, 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:19:23.344982 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Bound port: cbcf156d-db06-40df-9ad0-c2981b742561, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'c1fbdc79-842c-49eb-8789-2a1de3ab2b9f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 520, 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:19:23.345487 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.449879 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-9ef09ebb-5170-4536-a245-e07418896985 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.473517 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.486415 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-cfdf73dc-bc52-4371-bbc3-7f52ba5c80f6 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:23.488236 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-cfdf73dc-bc52-4371-bbc3-7f52ba5c80f6 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Synchronizing usage tracker for tenant:788050c389224d17b163f24bb67902bf on resource:network {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:23.492937 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.501123 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-af563873-d3e2-46a6-a8b8-da4ff5dbbbaa admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:23.508479 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] For port cbcf156d-db06-40df-9ad0-c2981b742561, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:19:23.517595 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] For port cbcf156d-db06-40df-9ad0-c2981b742561, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=cbcf156d-db06-40df-9ad0-c2981b742561,segment=NetworkSegment(c1fbdc79-842c-49eb-8789-2a1de3ab2b9f),segment_id=c1fbdc79-842c-49eb-8789-2a1de3ab2b9f)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:19:23.526119 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.528090 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f9d8d1f7-ec9c-42cd-84d1-0beadd04ddf3 req-3206d652-3917-4bdf-9817-eb000ef9f22c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:23.529786 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f9d8d1f7-ec9c-42cd-84d1-0beadd04ddf3 req-3206d652-3917-4bdf-9817-eb000ef9f22c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.1064446 May 24 11:19:23.530964 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:23.609184 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'SecurityGroup': ['ID=5928fed2-3207-44a0-91e4-485200bd1a27,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:23.613809 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:23.618242 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.622083 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Routers {'7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:19:23.622724 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:18:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:23.623602 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1792 time: 0.6374581 May 24 11:19:23.624682 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 56644) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:23.630844 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f9d8d1f7-ec9c-42cd-84d1-0beadd04ddf3 req-2caafa73-3529-4ffd-b252-8642c78af084 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0881445 May 24 11:19:23.634141 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-98fbf9b0-3fd0-4d82-9658-0b1c92adb6d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'security_group_rule': {'security_group_id': '5928fed2-3207-44a0-91e4-485200bd1a27', 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:23.639336 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-cfdf73dc-bc52-4371-bbc3-7f52ba5c80f6 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Unset dirty status for tenant:788050c389224d17b163f24bb67902bf on resource:network {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:23.640635 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-cfdf73dc-bc52-4371-bbc3-7f52ba5c80f6 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/networks?id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 HTTP/1.1" status: 200 len: 814 time: 0.3864417 May 24 11:19:23.643031 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-98fbf9b0-3fd0-4d82-9658-0b1c92adb6d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:23.649071 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-98fbf9b0-3fd0-4d82-9658-0b1c92adb6d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Usage tracker for resource:security_group_rule and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:23.653976 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-98fbf9b0-3fd0-4d82-9658-0b1c92adb6d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:23.668519 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-98fbf9b0-3fd0-4d82-9658-0b1c92adb6d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:23.671422 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-26d8190a-5314-4497-901a-2a95be56295a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/quotas/788050c389224d17b163f24bb67902bf HTTP/1.1" status: 200 len: 375 time: 0.0193460 May 24 11:19:23.749988 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-98fbf9b0-3fd0-4d82-9658-0b1c92adb6d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.769706 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d71c3a32-d104-46c1-8f8a-0c765c2e8c4a None None] Pushing event updated for resources: {'Port': ['ID=cbcf156d-db06-40df-9ad0-c2981b742561,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:23.781654 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-d71c3a32-d104-46c1-8f8a-0c765c2e8c4a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.251s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:23.815745 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-40941c8a-87c3-40b8-9817-9f8c1bdb56cf tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Synchronizing usage tracker for tenant:788050c389224d17b163f24bb67902bf on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:23.849032 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-40941c8a-87c3-40b8-9817-9f8c1bdb56cf tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Unset dirty status for tenant:788050c389224d17b163f24bb67902bf on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:23.850430 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-40941c8a-87c3-40b8-9817-9f8c1bdb56cf tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=788050c389224d17b163f24bb67902bf&fields=id HTTP/1.1" status: 200 len: 302 time: 0.1704805 May 24 11:19:23.852157 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-98fbf9b0-3fd0-4d82-9658-0b1c92adb6d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.858300 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-98fbf9b0-3fd0-4d82-9658-0b1c92adb6d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.859184 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] Set router 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:19:23.860581 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-7a2a1259-6355-4653-b689-09a8f3d73a5a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:23.969682 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98fbf9b0-3fd0-4d82-9658-0b1c92adb6d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=a4ae5d13-2ece-4d4b-b219-1c33a7af9246,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:23.972435 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-98fbf9b0-3fd0-4d82-9658-0b1c92adb6d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:23.998042 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-98fbf9b0-3fd0-4d82-9658-0b1c92adb6d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:23.999294 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-98fbf9b0-3fd0-4d82-9658-0b1c92adb6d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.3730605 May 24 11:19:24.002101 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 56648) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:24.009807 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'security_group_rule': {'security_group_id': '5928fed2-3207-44a0-91e4-485200bd1a27', 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:24.019220 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:24.027357 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Usage tracker for resource:security_group_rule and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:24.039559 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:24.050506 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:24.074004 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-aad41ef4-f8bc-4a5d-a616-92482338b795 service neutron] 10.222.0.18 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2061293 May 24 11:19:24.140830 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:24.295595 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'name': 'tempest-TestVolumeBootPattern-1369558850-router', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205'}], 'enable_snat': True}, 'gw_port_id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:19:08Z', 'updated_at': '2021-05-24T11:19:17Z', 'revision_number': 4, 'project_id': '672c5f4985084a8492a86667aceb0768', 'gw_port': {'id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:68:b5:40', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 403, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:19:10Z', 'updated_at': '2021-05-24T11:19:21Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'cbcf156d-db06-40df-9ad0-c2981b742561', 'name': '', 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'mac_address': 'fa:16:3e:82:35:f2', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'device_owner': 'network:router_interface', 'standard_attr_id': 404, 'fixed_ips': [{'subnet_id': 'a25edff7-c0c6-44d4-ad1c-a38e28a26a35', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:19:16Z', 'updated_at': '2021-05-24T11:19:23Z', 'revision_number': 4, 'project_id': '672c5f4985084a8492a86667aceb0768', 'subnets': [{'id': 'a25edff7-c0c6-44d4-ad1c-a38e28a26a35', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:24.302257 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9cb2c5a8-85cd-44e2-b297-10cf80d78b02 None None] The sync data for ids ['7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'] in dv-cinder-784092-9-iscsi is: [{'id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'name': 'tempest-TestVolumeBootPattern-1369558850-router', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205'}], 'enable_snat': True}, 'gw_port_id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:19:08Z', 'updated_at': '2021-05-24T11:19:17Z', 'revision_number': 4, 'project_id': '672c5f4985084a8492a86667aceb0768', 'gw_port': {'id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:68:b5:40', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 403, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:19:10Z', 'updated_at': '2021-05-24T11:19:21Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'cbcf156d-db06-40df-9ad0-c2981b742561', 'name': '', 'network_id': 'e6b6c00d-d044-4bc7-92ce-02e445f1f072', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'mac_address': 'fa:16:3e:82:35:f2', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'device_owner': 'network:router_interface', 'standard_attr_id': 404, 'fixed_ips': [{'subnet_id': 'a25edff7-c0c6-44d4-ad1c-a38e28a26a35', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:19:16Z', 'updated_at': '2021-05-24T11:19:23Z', 'revision_number': 4, 'project_id': '672c5f4985084a8492a86667aceb0768', 'subnets': [{'id': 'a25edff7-c0c6-44d4-ad1c-a38e28a26a35', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:19:24.315437 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:19:24.345469 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-c6f36294-96d6-4323-b35d-89a4da2086cc service neutron] 10.222.0.18 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2623758 May 24 11:19:24.362251 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:24.367624 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1434483-e505-428b-9fac-60c463e6fcf2 req-62564daa-0e02-4980-ba79-81a6cc8c3e14 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:24.372887 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:24.376260 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e1434483-e505-428b-9fac-60c463e6fcf2 req-62564daa-0e02-4980-ba79-81a6cc8c3e14 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 946 time: 0.1261013 May 24 11:19:24.396143 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e17fdf0b-45af-4416-9fdb-45002e2d95a7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:24.429073 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2745fb47-e8ff-4225-ad60-2cca0ea067ca req-5c918daf-127a-4bd6-b46b-89ccb7b6c627 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:24.429705 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2745fb47-e8ff-4225-ad60-2cca0ea067ca req-5c918daf-127a-4bd6-b46b-89ccb7b6c627 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.93 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 946 time: 0.3266511 May 24 11:19:24.435756 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=868dd259-62aa-4b36-931e-094296457fd3,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:24.438963 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:24.442354 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:24.446391 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.4404533 May 24 11:19:24.461460 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e1434483-e505-428b-9fac-60c463e6fcf2 req-d17076e6-7ff9-491b-ad9e-f2e76cd80561 tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/security-groups?id=57604184-194b-41df-8615-c443c9e6dfa4&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0839951 May 24 11:19:24.461971 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56654) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:24.476198 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'security_group_rule': {'security_group_id': '5928fed2-3207-44a0-91e4-485200bd1a27', 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:24.482639 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:24.490204 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device a4cc0b23-ca4a-4a18-9cee-ce596875baf8 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:24.490521 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Usage tracker for resource:security_group_rule and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:24.494477 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:24.502730 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:24.530625 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:24.592975 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port a4cc0b23-ca4a-4a18-9cee-ce596875baf8 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:24.593648 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:24.659545 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [req-2745fb47-e8ff-4225-ad60-2cca0ea067ca req-0759abe1-2c28-411e-985a-24b91bb60eea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Deleting port 11c2ccd2-652a-4862-8c69-587b35b23237 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:19:24.659895 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-2745fb47-e8ff-4225-ad60-2cca0ea067ca req-0759abe1-2c28-411e-985a-24b91bb60eea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:24.700157 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-05d33223-48e8-45d8-9f0e-49ba26966afb req-b19e8944-7086-42be-b013-adf7b76c3336 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 210 time: 0.0910203 May 24 11:19:24.805864 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:24.822008 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:24.852593 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:24.865784 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:24.867216 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.4042222 May 24 11:19:24.871926 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56672) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:24.876249 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=9d49876f-a49f-4039-8626-98159664c44c,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:24.882671 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'security_group_rule': {'security_group_id': '5928fed2-3207-44a0-91e4-485200bd1a27', 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:24.890878 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:24.891968 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:24.898009 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Usage tracker for resource:security_group_rule and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:24.902224 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:24.911430 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:24.928025 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-2745fb47-e8ff-4225-ad60-2cca0ea067ca req-0759abe1-2c28-411e-985a-24b91bb60eea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] For port 11c2ccd2-652a-4862-8c69-587b35b23237, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=11c2ccd2-652a-4862-8c69-587b35b23237,segment=NetworkSegment(f3295bfe-6894-409e-b0cc-96311bd90247),segment_id=f3295bfe-6894-409e-b0cc-96311bd90247)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:24.928674 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-2745fb47-e8ff-4225-ad60-2cca0ea067ca req-0759abe1-2c28-411e-985a-24b91bb60eea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:24.941067 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.028853 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.039572 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.043348 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-1af527fe-5960-42fd-9741-cb37b0e9b074 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:25.095185 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=22af9505-c46f-47f2-b6b9-2aa84aa7b429,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:25.099331 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:25.104435 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.106472 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.2319548 May 24 11:19:25.109238 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56678) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:25.119872 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'security_group_rule': {'security_group_id': '5928fed2-3207-44a0-91e4-485200bd1a27', 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:25.146528 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:25.151039 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Usage tracker for resource:security_group_rule and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:25.154484 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:25.163067 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:25.188849 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.securitygroups_db [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Project 9a8d61f79bbc496ca30e390fd9b5839e added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 24 11:19:25.197064 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.288795 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [req-2745fb47-e8ff-4225-ad60-2cca0ea067ca req-0759abe1-2c28-411e-985a-24b91bb60eea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Calling delete_port for 11c2ccd2-652a-4862-8c69-587b35b23237 owned by compute:nova {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:19:25.360018 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.361549 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e9260cc-672a-48a1-88f5-da5977f8b9fa req-be2829ad-2a9c-4d68-8101-c0d9d3af2515 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:25.369695 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6e9260cc-672a-48a1-88f5-da5977f8b9fa req-be2829ad-2a9c-4d68-8101-c0d9d3af2515 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.4449520 May 24 11:19:25.369862 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.378998 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:25.414170 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=52ef7a16-6aac-497c-a210-a5ddd9b7cf8f,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:25.418377 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:25.435582 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6e9260cc-672a-48a1-88f5-da5977f8b9fa req-30f6084c-e6d4-4037-8177-7ce2108e8ee4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0647368 May 24 11:19:25.436961 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-35b17a0a-3b85-466f-b357-04ddf5df68d3 req-764013aa-755c-479c-87be-570775b95fc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:25.439453 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-35b17a0a-3b85-466f-b357-04ddf5df68d3 req-764013aa-755c-479c-87be-570775b95fc3 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 945 time: 0.1611495 May 24 11:19:25.441272 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.442724 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.3323247 May 24 11:19:25.445937 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 56684) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:25.469123 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'security_group_rule': {'security_group_id': '5928fed2-3207-44a0-91e4-485200bd1a27', 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:25.479933 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:25.488045 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Usage tracker for resource:security_group_rule and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:19:25.498382 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:19:25.513526 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:25.527036 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-35b17a0a-3b85-466f-b357-04ddf5df68d3 req-3578312d-c180-46ee-8f6f-8238d4cd0f6f tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0768871 May 24 11:19:25.531259 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-2745fb47-e8ff-4225-ad60-2cca0ea067ca req-0759abe1-2c28-411e-985a-24b91bb60eea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.537771 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.securitygroups_db [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Project 9a8d61f79bbc496ca30e390fd9b5839e added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 24 11:19:25.546367 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-10828226-3c82-4135-a587-f7166cfa0b9a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:25.556556 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.693351 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0759abe1-2c28-411e-985a-24b91bb60eea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Pushing event deleted for resources: {'Port': ['ID=11c2ccd2-652a-4862-8c69-587b35b23237,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:25.742185 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0759abe1-2c28-411e-985a-24b91bb60eea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.198s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:25.752232 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d902a7a2-2ea1-42cc-ba89-aa6ad6dad9ca req-7dbcf789-15e6-4372-a32a-aca01d22bc9c tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.18 "GET /v2.0/ports?device_id=07f504cc-a04d-4738-8be7-ff333195f9d9 HTTP/1.1" status: 200 len: 210 time: 0.0599613 May 24 11:19:25.769533 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.789158 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.824400 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [req-2745fb47-e8ff-4225-ad60-2cca0ea067ca req-0759abe1-2c28-411e-985a-24b91bb60eea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] No DVR routers for this DVR port 11c2ccd2-652a-4862-8c69-587b35b23237 on host hv-cinder-78409 {{(pid=91820) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:19:25.827314 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-2745fb47-e8ff-4225-ad60-2cca0ea067ca req-0759abe1-2c28-411e-985a-24b91bb60eea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.830274 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '07f504cc-a04d-4738-8be7-ff333195f9d9', 'name': 'network-vif-deleted', 'tag': '11c2ccd2-652a-4862-8c69-587b35b23237'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:19:25.840861 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.855395 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2745fb47-e8ff-4225-ad60-2cca0ea067ca req-0759abe1-2c28-411e-985a-24b91bb60eea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] 10.222.0.93 "DELETE /v2.0/ports/11c2ccd2-652a-4862-8c69-587b35b23237 HTTP/1.1" status: 204 len: 173 time: 1.4149158 May 24 11:19:25.855877 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-0f775df8-43ed-4fad-8048-1ceefd669a56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:25.896459 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-15cc1d5c-fc4a-4eb4-aec7-80ade949da0c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.934962 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '07f504cc-a04d-4738-8be7-ff333195f9d9', 'name': 'network-vif-deleted', 'tag': '11c2ccd2-652a-4862-8c69-587b35b23237', 'status': 'completed', 'code': 200} May 24 11:19:25.946911 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=c672b96f-ed61-4538-b1a1-62eabd876c86,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:25.956383 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.101s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:25.967474 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-73ee2510-5584-4acb-862f-29649bf87cc2 req-e1ccc07e-1d30-420a-a332-042f61ca9c02 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 210 time: 0.0683596 May 24 11:19:25.968229 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:25.973250 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.5222826 May 24 11:19:26.015071 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a40e20e4-42ea-4015-b5f3-a38cbc093b81 req-19f4d60a-cd16-434f-baf5-eecddc3d4669 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:26.016705 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a40e20e4-42ea-4015-b5f3-a38cbc093b81 req-19f4d60a-cd16-434f-baf5-eecddc3d4669 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.93 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 946 time: 0.2259655 May 24 11:19:26.234773 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-a40e20e4-42ea-4015-b5f3-a38cbc093b81 req-a91ab59f-d2c2-47f8-8b81-0cbdd43a2ba4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Deleting port 692ab816-0fdb-448b-9b27-c44c0a457f4d {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:19:26.235234 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40e20e4-42ea-4015-b5f3-a38cbc093b81 req-a91ab59f-d2c2-47f8-8b81-0cbdd43a2ba4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:26.467334 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-a40e20e4-42ea-4015-b5f3-a38cbc093b81 req-a91ab59f-d2c2-47f8-8b81-0cbdd43a2ba4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] For port 692ab816-0fdb-448b-9b27-c44c0a457f4d, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=692ab816-0fdb-448b-9b27-c44c0a457f4d,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:26.467766 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40e20e4-42ea-4015-b5f3-a38cbc093b81 req-a91ab59f-d2c2-47f8-8b81-0cbdd43a2ba4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:26.511492 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device cbcf156d-db06-40df-9ad0-c2981b742561 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:26.681296 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-a40e20e4-42ea-4015-b5f3-a38cbc093b81 req-a91ab59f-d2c2-47f8-8b81-0cbdd43a2ba4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Calling delete_port for 692ab816-0fdb-448b-9b27-c44c0a457f4d owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:19:26.842040 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-95e8a975-74b7-4940-89a4-a31baca0fbce tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:26.843616 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-95e8a975-74b7-4940-89a4-a31baca0fbce tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.93 "GET /v2.0/networks?id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 HTTP/1.1" status: 200 len: 814 time: 0.5957682 May 24 11:19:26.870836 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port cbcf156d-db06-40df-9ad0-c2981b742561 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:19:26.876617 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Request body: {'port': {'device_id': 'fa7a9207-daa9-4469-9eec-51039940736a', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'admin_state_up': True, 'tenant_id': '788050c389224d17b163f24bb67902bf'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:26.885759 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40e20e4-42ea-4015-b5f3-a38cbc093b81 req-a91ab59f-d2c2-47f8-8b81-0cbdd43a2ba4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:26.890470 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port cbcf156d-db06-40df-9ad0-c2981b742561 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:26.891081 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:26.921059 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:26.993530 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['02210ade-8c60-41b6-aa82-a104d4fc0447', '692ab816-0fdb-448b-9b27-c44c0a457f4d'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:19:27.028428 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a5211e7f-e7b8-42d5-9567-6c103df53541 req-85083fc1-4d35-4d4f-a857-a8e47ba2f624 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.1092076 May 24 11:19:27.127346 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a91ab59f-d2c2-47f8-8b81-0cbdd43a2ba4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Pushing event deleted for resources: {'Port': ['ID=692ab816-0fdb-448b-9b27-c44c0a457f4d,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:27.147251 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a91ab59f-d2c2-47f8-8b81-0cbdd43a2ba4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.226s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:27.305743 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:27.319755 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 692ab816-0fdb-448b-9b27-c44c0a457f4d no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:19:27.334489 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:27.367322 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Made reservation on behalf of 788050c389224d17b163f24bb67902bf for: {'port': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:19:27.371295 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:27.403820 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 692ab816-0fdb-448b-9b27-c44c0a457f4d {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:19:27.404211 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 692ab816-0fdb-448b-9b27-c44c0a457f4d not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:19:27.404640 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 11c2ccd2-652a-4862-8c69-587b35b23237 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:19:27.491824 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-a40e20e4-42ea-4015-b5f3-a38cbc093b81 req-a91ab59f-d2c2-47f8-8b81-0cbdd43a2ba4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] No DVR routers for this DVR port 692ab816-0fdb-448b-9b27-c44c0a457f4d on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:19:27.552117 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a40e20e4-42ea-4015-b5f3-a38cbc093b81 req-a91ab59f-d2c2-47f8-8b81-0cbdd43a2ba4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:27.565569 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 11c2ccd2-652a-4862-8c69-587b35b23237 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:19:27.566527 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 11c2ccd2-652a-4862-8c69-587b35b23237 not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:19:27.568140 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:27.570041 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:27.621721 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'ef0f174b-4c60-4917-8c38-60dd5c2c0add', 'name': 'network-vif-deleted', 'tag': '692ab816-0fdb-448b-9b27-c44c0a457f4d'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:19:27.650986 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a40e20e4-42ea-4015-b5f3-a38cbc093b81 req-a91ab59f-d2c2-47f8-8b81-0cbdd43a2ba4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.93 "DELETE /v2.0/ports/692ab816-0fdb-448b-9b27-c44c0a457f4d HTTP/1.1" status: 204 len: 173 time: 1.6277618 May 24 11:19:27.724241 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port cbcf156d-db06-40df-9ad0-c2981b742561, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=cbcf156d-db06-40df-9ad0-c2981b742561,segment=NetworkSegment(c1fbdc79-842c-49eb-8789-2a1de3ab2b9f),segment_id=c1fbdc79-842c-49eb-8789-2a1de3ab2b9f)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:27.732102 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:27.740064 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:27.768302 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1027c1c-17a7-48f1-8ed2-ee888dfcfa9b req-b23db3e5-b5c2-4d2c-a875-07e0d2e2f84a tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:27.781299 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d1027c1c-17a7-48f1-8ed2-ee888dfcfa9b req-b23db3e5-b5c2-4d2c-a875-07e0d2e2f84a tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 945 time: 0.7711074 May 24 11:19:27.808897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3db4ddc7-e6e0-4147-8f22-917f18583902 req-2dea7b8a-b9b2-4a2a-bfe9-abad42f857ef tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:27.823441 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3db4ddc7-e6e0-4147-8f22-917f18583902 req-2dea7b8a-b9b2-4a2a-bfe9-abad42f857ef tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 945 time: 0.6951215 May 24 11:19:27.823639 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=cbcf156d-db06-40df-9ad0-c2981b742561,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:27.835812 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:27.841979 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-42e128d0-871e-4441-8f4a-613f8f419ca5 req-9eeaa6c2-017b-4a5a-a9dc-d782245c40da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 210 time: 0.1583285 May 24 11:19:28.001473 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'ef0f174b-4c60-4917-8c38-60dd5c2c0add', 'name': 'network-vif-deleted', 'tag': '692ab816-0fdb-448b-9b27-c44c0a457f4d', 'status': 'completed', 'code': 200} May 24 11:19:28.005948 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Allocated IP 172.20.0.6 (0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397/6a80c579-20d5-46c7-8c07-6438f1367afd/e0dbce52-01cb-4042-b340-ea0a6b7da010) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:19:28.125173 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3db4ddc7-e6e0-4147-8f22-917f18583902 req-cb77c98b-7487-40d9-826c-06a66f0f4d04 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2935410 May 24 11:19:28.226534 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-d1027c1c-17a7-48f1-8ed2-ee888dfcfa9b req-fdac78f2-de00-41b5-8091-2be92a4b3c51 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Deleting port 516f8f18-b5b5-4ded-bdfd-c420f5651d69 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:19:28.227758 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-d1027c1c-17a7-48f1-8ed2-ee888dfcfa9b req-fdac78f2-de00-41b5-8091-2be92a4b3c51 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:28.500410 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device cbcf156d-db06-40df-9ad0-c2981b742561 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:28.513258 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ee220fb0-1b00-49f7-8691-98bf29736778 req-b51aeb0f-1f2b-48f5-a266-0c8217f9c9b7 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.1259108 May 24 11:19:28.582479 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-d1027c1c-17a7-48f1-8ed2-ee888dfcfa9b req-fdac78f2-de00-41b5-8091-2be92a4b3c51 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] For port 516f8f18-b5b5-4ded-bdfd-c420f5651d69, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=516f8f18-b5b5-4ded-bdfd-c420f5651d69,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:28.582752 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-d1027c1c-17a7-48f1-8ed2-ee888dfcfa9b req-fdac78f2-de00-41b5-8091-2be92a4b3c51 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:28.731299 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port cbcf156d-db06-40df-9ad0-c2981b742561 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:28.731681 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:28.781440 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-d1027c1c-17a7-48f1-8ed2-ee888dfcfa9b req-fdac78f2-de00-41b5-8091-2be92a4b3c51 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Calling delete_port for 516f8f18-b5b5-4ded-bdfd-c420f5651d69 owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:19:28.847962 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:28.919269 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Transition to ACTIVE for port object e0dbce52-01cb-4042-b340-ea0a6b7da010 will not be triggered until provisioned by entity DHCP. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:19:28.998813 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:29.005779 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:29.025021 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:29.098997 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:19:29.115311 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:19:29.118657 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:19:29.141383 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-d1027c1c-17a7-48f1-8ed2-ee888dfcfa9b req-fdac78f2-de00-41b5-8091-2be92a4b3c51 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:29.173980 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['45911d6f-9595-40c5-8b29-75259c824bfe', '159fe3b9-426c-472e-9870-2ead1792535a', '516f8f18-b5b5-4ded-bdfd-c420f5651d69'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:19:29.181492 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:29.375654 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event updated for resources: {'Port': ['ID=e0dbce52-01cb-4042-b340-ea0a6b7da010,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:29.571344 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.546s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:29.599986 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:29.601121 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:29.684260 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-c5daa1a2-1377-4d61-abed-5676b74f0956 None None] Provisioning for port e0dbce52-01cb-4042-b340-ea0a6b7da010 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:19:29.719668 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-c5daa1a2-1377-4d61-abed-5676b74f0956 None None] Provisioning complete for port e0dbce52-01cb-4042-b340-ea0a6b7da010 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:29.720337 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c5daa1a2-1377-4d61-abed-5676b74f0956 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:29.762298 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 2.9059424 May 24 11:19:29.785596 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-110107ed-eb98-4714-bb77-60bbe1077047 req-85c12217-eff5-4516-bc16-49c6c21ffaaa tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.4340932 May 24 11:19:29.936746 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fdac78f2-de00-41b5-8091-2be92a4b3c51 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Pushing event deleted for resources: {'Port': ['ID=516f8f18-b5b5-4ded-bdfd-c420f5651d69,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:29.965591 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-fdac78f2-de00-41b5-8091-2be92a4b3c51 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.784s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:30.089635 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-c5daa1a2-1377-4d61-abed-5676b74f0956 None None] Port e0dbce52-01cb-4042-b340-ea0a6b7da010 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:19:30.152540 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-d1027c1c-17a7-48f1-8ed2-ee888dfcfa9b req-fdac78f2-de00-41b5-8091-2be92a4b3c51 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] No DVR routers for this DVR port 516f8f18-b5b5-4ded-bdfd-c420f5651d69 on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:19:30.156234 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-d1027c1c-17a7-48f1-8ed2-ee888dfcfa9b req-fdac78f2-de00-41b5-8091-2be92a4b3c51 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:30.159166 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '70697b65-6432-4d86-8536-934a7bf644ad', 'name': 'network-vif-deleted', 'tag': '516f8f18-b5b5-4ded-bdfd-c420f5651d69'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:19:30.189839 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d1027c1c-17a7-48f1-8ed2-ee888dfcfa9b req-fdac78f2-de00-41b5-8091-2be92a4b3c51 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "DELETE /v2.0/ports/516f8f18-b5b5-4ded-bdfd-c420f5651d69 HTTP/1.1" status: 204 len: 173 time: 2.3458185 May 24 11:19:30.207451 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-16cfdf0e-9ab2-454b-a3bf-57331d73da24 req-99f6816c-7575-48c5-9ccf-ed2dbc297db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:30.214950 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-16cfdf0e-9ab2-454b-a3bf-57331d73da24 req-99f6816c-7575-48c5-9ccf-ed2dbc297db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 931 time: 0.4680333 May 24 11:19:30.305277 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-16cfdf0e-9ab2-454b-a3bf-57331d73da24 req-70c9e973-4f8c-4553-8e4a-dd0807ff2d37 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0882511 May 24 11:19:30.426773 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '70697b65-6432-4d86-8536-934a7bf644ad', 'name': 'network-vif-deleted', 'tag': '516f8f18-b5b5-4ded-bdfd-c420f5651d69', 'status': 'completed', 'code': 200} May 24 11:19:30.460937 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5f52c10c-fa5c-4d63-bd6e-513806b48017 req-066483bd-19f3-4dd4-8d9d-4420718529cf tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.2472439 May 24 11:19:31.001353 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-be3a96c6-6295-4d55-ba80-3d916ce1b1ef req-3cf48649-eb21-4d50-a6b9-7b6c03c4b04d tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Synchronizing usage tracker for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:31.026671 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-be3a96c6-6295-4d55-ba80-3d916ce1b1ef req-3cf48649-eb21-4d50-a6b9-7b6c03c4b04d tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Unset dirty status for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:31.028252 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-be3a96c6-6295-4d55-ba80-3d916ce1b1ef req-3cf48649-eb21-4d50-a6b9-7b6c03c4b04d tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0988731 May 24 11:19:31.172510 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 516f8f18-b5b5-4ded-bdfd-c420f5651d69 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:19:31.225377 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 516f8f18-b5b5-4ded-bdfd-c420f5651d69 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:19:31.226943 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 516f8f18-b5b5-4ded-bdfd-c420f5651d69 not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:19:31.558062 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e076dd94-c357-4a18-9e78-f1c465cc0a71 req-87a456c8-5acc-449b-a410-4d3ad2e6d418 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:31.558808 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-e076dd94-c357-4a18-9e78-f1c465cc0a71 req-87a456c8-5acc-449b-a410-4d3ad2e6d418 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Synchronizing usage tracker for tenant:788050c389224d17b163f24bb67902bf on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:31.598239 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-e076dd94-c357-4a18-9e78-f1c465cc0a71 req-87a456c8-5acc-449b-a410-4d3ad2e6d418 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Unset dirty status for tenant:788050c389224d17b163f24bb67902bf on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:31.600619 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e076dd94-c357-4a18-9e78-f1c465cc0a71 req-87a456c8-5acc-449b-a410-4d3ad2e6d418 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 931 time: 0.1489606 May 24 11:19:31.650896 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-e076dd94-c357-4a18-9e78-f1c465cc0a71 req-0d3a4263-a159-42c3-9119-7fee5eeca53c tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Synchronizing usage tracker for tenant:788050c389224d17b163f24bb67902bf on resource:security_group {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:31.670968 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-e076dd94-c357-4a18-9e78-f1c465cc0a71 req-0d3a4263-a159-42c3-9119-7fee5eeca53c tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Unset dirty status for tenant:788050c389224d17b163f24bb67902bf on resource:security_group {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:31.672209 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e076dd94-c357-4a18-9e78-f1c465cc0a71 req-0d3a4263-a159-42c3-9119-7fee5eeca53c tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0645170 May 24 11:19:31.704280 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-a5616ded-e09a-4f01-9e15-55dc0bef7d0b req-278f203a-a586-43ca-82d5-c99bf8889a2d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Synchronizing usage tracker for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:31.719774 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-a5616ded-e09a-4f01-9e15-55dc0bef7d0b req-278f203a-a586-43ca-82d5-c99bf8889a2d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Unset dirty status for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:31.720888 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a5616ded-e09a-4f01-9e15-55dc0bef7d0b req-278f203a-a586-43ca-82d5-c99bf8889a2d tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0940011 May 24 11:19:32.218811 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6b9bc369-3150-4077-9cc1-89c272fdf540 req-940a61f5-f93a-4eee-b8ce-1e8c74dbc6b2 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0597997 May 24 11:19:32.981134 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7dfd4ee-91ca-4938-b6ec-4115adffbd1e req-d8de9005-91a9-4f58-b7b9-f67efd503759 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:32.984758 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a7dfd4ee-91ca-4938-b6ec-4115adffbd1e req-d8de9005-91a9-4f58-b7b9-f67efd503759 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 931 time: 0.1499562 May 24 11:19:33.018314 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-36e10d48-170d-4fa4-988e-c17119076764 req-58fb9c9d-f717-4b77-b28f-a77d6e63c02b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0834317 May 24 11:19:33.070181 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a7dfd4ee-91ca-4938-b6ec-4115adffbd1e req-1756f719-e170-4ec5-af5c-f9e58e100fab tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0774434 May 24 11:19:33.397905 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-08b0f18c-4d29-4610-9b19-c1f32d8c0a62 req-bdd61a36-040d-428f-a643-49b2802122d1 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0605066 May 24 11:19:34.313647 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9ce70e4-3e4c-46ec-9c6d-4a37f14fb364 req-e826704b-3ab0-4849-a71d-be9de81983ae tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:34.322982 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e9ce70e4-3e4c-46ec-9c6d-4a37f14fb364 req-e826704b-3ab0-4849-a71d-be9de81983ae tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 931 time: 0.1097040 May 24 11:19:34.329910 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-97e39347-c8f3-446f-9305-56d8fec40c63 req-2e37ca37-0a91-4a26-be41-804852059401 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0708144 May 24 11:19:34.373874 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e9ce70e4-3e4c-46ec-9c6d-4a37f14fb364 req-86b48d4e-eae3-491c-bac1-95071cf73bc3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0461624 May 24 11:19:34.577752 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b86be15f-22a8-4544-956c-58290a440cb9 req-fea2ca59-3568-4f53-a5ea-dc35ea91ec6e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0551023 May 24 11:19:35.505673 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2ec51eba-849d-4105-acea-b35c670e5705 req-d9bf927b-d7c5-45bc-8953-691403bdbaac tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0443470 May 24 11:19:35.684656 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72a33fe0-904f-4533-b6a9-f4d813edd6be req-f447e6db-48ae-4887-be32-776ac70dbc77 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:35.698228 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-72a33fe0-904f-4533-b6a9-f4d813edd6be req-f447e6db-48ae-4887-be32-776ac70dbc77 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 931 time: 0.1486669 May 24 11:19:35.757206 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-72a33fe0-904f-4533-b6a9-f4d813edd6be req-7b9b7bee-42d1-451a-a798-26c5e9da9f84 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0577521 May 24 11:19:35.781115 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-365140d4-e364-455c-954e-4350a7517b6d req-6072256c-59e4-4e14-b8b0-0eeea056e75d tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0622435 May 24 11:19:36.697176 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e84489fd-6caa-4f3a-ba97-e7e35133cf3f req-f05bdfc2-6903-4274-a6ef-fd04330ba7d8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0489893 May 24 11:19:36.994549 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-177ed31a-e189-4897-b066-72f6a80d94ba req-3e154c67-16fd-4a52-ad25-d92628634c74 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0626867 May 24 11:19:37.004437 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-36ec5394-cf17-4d0c-810d-cfd837b69637 req-d3cc17b6-1605-4b1d-99c7-4ad923328ac5 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:37.006019 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-36ec5394-cf17-4d0c-810d-cfd837b69637 req-d3cc17b6-1605-4b1d-99c7-4ad923328ac5 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 931 time: 0.1118577 May 24 11:19:37.069947 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-36ec5394-cf17-4d0c-810d-cfd837b69637 req-3b9e88ca-0024-4ca5-be51-c95c1822b089 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0580106 May 24 11:19:37.898696 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6b48a9af-2ab5-414b-a548-505c4f462d72 req-62b17b53-cf40-4cb0-8460-d7f2d4bc8883 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0510066 May 24 11:19:38.204812 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-20eab56b-f915-4686-894d-32356dfbc652 req-49d04305-e7ba-45ca-8166-e84643aab2ce tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0538406 May 24 11:19:38.324611 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f25a905b-e93d-4c2f-881f-702b990d413b req-55822483-078a-4197-ac47-de98b0ed5706 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:38.326097 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f25a905b-e93d-4c2f-881f-702b990d413b req-55822483-078a-4197-ac47-de98b0ed5706 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 931 time: 0.1218536 May 24 11:19:38.382703 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f25a905b-e93d-4c2f-881f-702b990d413b req-1d1f3403-deb7-45e7-96f7-acf172f37aab tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0503521 May 24 11:19:39.095230 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a0ba387c-3e10-41c8-84d9-8b3c064e6691 req-696306f7-59c5-420e-8d80-ff2dee2628ea tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0502651 May 24 11:19:39.438408 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b19871fd-0d8e-4df4-8d1c-7b272a925563 req-79aaeead-c6ae-432a-baa1-a63ea4eabf0c tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0802205 May 24 11:19:39.602970 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bafa3761-1e3e-4f92-bdaa-dfda8f507fe8 req-9b8bae41-5c57-4480-8321-11e51acb0b1d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:39.604476 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-bafa3761-1e3e-4f92-bdaa-dfda8f507fe8 req-9b8bae41-5c57-4480-8321-11e51acb0b1d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 931 time: 0.1115265 May 24 11:19:39.663322 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-bafa3761-1e3e-4f92-bdaa-dfda8f507fe8 req-51b74458-64e5-4c54-8be4-89c594c5951c tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0504322 May 24 11:19:40.353401 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-444e0b22-b21c-4231-9029-12956c0bfe4b req-2389179e-ba48-4343-863f-1c90faf05b4e tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0617700 May 24 11:19:40.652039 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-68708e8f-45d8-4129-9cc4-e4f60aaf79a8 req-ca535682-abd8-4797-8ad4-90e15633a1ab tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0763581 May 24 11:19:40.903211 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db744153-746e-40d5-83d4-a345c97af268 req-38024d61-d133-49c6-9a41-5033105d155f tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:40.905682 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-db744153-746e-40d5-83d4-a345c97af268 req-38024d61-d133-49c6-9a41-5033105d155f tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 931 time: 0.0933070 May 24 11:19:40.954264 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-db744153-746e-40d5-83d4-a345c97af268 req-6aec9736-4411-48f7-9bbe-f17d6c57521b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0422585 May 24 11:19:41.524389 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-00870fac-8dde-4eac-bce7-17a72d6035f0 req-bf2731bf-810d-4ccb-805c-e8df8eedf8cd tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0430722 May 24 11:19:41.808078 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-35497db9-b856-4458-a1df-7eadb912fc3f req-1851f146-50f2-4dd9-b751-a530c106069e tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0467267 May 24 11:19:42.316853 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8bb7ee99-1086-44fd-9296-7b9a9a7568bb req-01f18c3f-ab14-46ef-9224-34d7a7adbb38 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:42.318241 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8bb7ee99-1086-44fd-9296-7b9a9a7568bb req-01f18c3f-ab14-46ef-9224-34d7a7adbb38 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 931 time: 0.1010590 May 24 11:19:42.382003 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8bb7ee99-1086-44fd-9296-7b9a9a7568bb req-44671592-3a3e-43ca-9fb4-335a0db1ee82 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0540960 May 24 11:19:42.712163 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8e6bb634-92a3-44f4-9b74-80ccaab472ec req-3df29171-e50f-41a8-ad49-72cbf645e193 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0435696 May 24 11:19:42.961669 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-20da6a62-e6c5-45a2-808b-5ce2a973748a req-06c3958f-ee34-43bf-82a0-bf0a5573d897 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0439310 May 24 11:19:43.586104 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7e56b270-fbbb-4cee-8b44-c9e2789d143c req-28f0b374-6e7a-4508-acd7-de26c11bccff tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:43.587665 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7e56b270-fbbb-4cee-8b44-c9e2789d143c req-28f0b374-6e7a-4508-acd7-de26c11bccff tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 931 time: 0.0939219 May 24 11:19:43.636173 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7e56b270-fbbb-4cee-8b44-c9e2789d143c req-65ff44a3-f4e6-4d3f-a1b7-ea778d36839e tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0427859 May 24 11:19:43.877651 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Request body: {'port': {'device_id': 'fa7a9207-daa9-4469-9eec-51039940736a', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:43.927881 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-94cb268e-230a-4284-9f79-2d24bc7aae8b req-8e2623dd-70ce-486b-bb13-070602baa7d2 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=ef0f174b-4c60-4917-8c38-60dd5c2c0add HTTP/1.1" status: 200 len: 210 time: 0.0887821 May 24 11:19:44.037603 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:44.173553 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-659ffb1f-9b11-498e-a286-0e2752b46c76 req-3ff334a4-1efb-4c2f-addc-10bd1feb98f2 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=70697b65-6432-4d86-8536-934a7bf644ad HTTP/1.1" status: 200 len: 210 time: 0.0456941 May 24 11:19:44.440357 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id e0dbce52-01cb-4042-b340-ea0a6b7da010 {{(pid=91820) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:19:44.447949 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:44.471671 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Transition to ACTIVE for port object e0dbce52-01cb-4042-b340-ea0a6b7da010 will not be triggered until provisioned by entity L2. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:19:44.476161 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:44.482397 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6f5b3099-2d7b-4ef1-b9dc-b625681b32f4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:44.840946 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Pushing event updated for resources: {'Port': ['ID=e0dbce52-01cb-4042-b340-ea0a6b7da010,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:44.869883 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.388s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:44.960021 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Attempting to bind port e0dbce52-01cb-4042-b340-ea0a6b7da010 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:19:44.961737 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Attempting to bind port e0dbce52-01cb-4042-b340-ea0a6b7da010 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:19:44.962445 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Attempting to bind port e0dbce52-01cb-4042-b340-ea0a6b7da010 on network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:19:44.978491 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Port e0dbce52-01cb-4042-b340-ea0a6b7da010 on network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:19:44.979477 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Attempting to bind port e0dbce52-01cb-4042-b340-ea0a6b7da010 on network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:19:44.992841 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 19, 22), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:19:44.993534 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Checking segment: {'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:19:44.994176 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Bound using segment: {'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:19:44.994836 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Bound port: e0dbce52-01cb-4042-b340-ea0a6b7da010, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '666aa9e5-b138-4841-9cc4-ecbe52ac33c1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 539, 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397'}}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:19:44.995589 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:45.098194 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6af3317d-50b4-425a-8f13-415f72d570a2 req-56481cd3-eda7-4fe5-9db7-63ddd70d8dfb tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:45.099743 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6af3317d-50b4-425a-8f13-415f72d570a2 req-56481cd3-eda7-4fe5-9db7-63ddd70d8dfb tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 943 time: 0.2976403 May 24 11:19:45.215759 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6af3317d-50b4-425a-8f13-415f72d570a2 req-9ff86ee0-67c4-4d8a-ada0-bbd2b542928a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1067045 May 24 11:19:45.223132 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] For port e0dbce52-01cb-4042-b340-ea0a6b7da010, host hv-cinder-78409, cleared binding levels {{(pid=91820) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:19:45.228462 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] For port e0dbce52-01cb-4042-b340-ea0a6b7da010, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91820) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:19:45.239723 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:45.247493 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f4bd8a23-a803-41b7-804d-6a4aa2dee1ef service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:45.342634 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:45.343605 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:45.345906 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] 10.222.0.93 "PUT /v2.0/ports/e0dbce52-01cb-4042-b340-ea0a6b7da010 HTTP/1.1" status: 200 len: 1093 time: 1.4723434 May 24 11:19:45.346387 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:19:45.384877 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Pushing event updated for resources: {'Port': ['ID=e0dbce52-01cb-4042-b340-ea0a6b7da010,revision_number=3']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:45.398915 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b2db1e9b-9f0d-485e-bbd6-3f1b7a4cb228 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.151s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:45.479965 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'status': 'completed', 'code': 200} May 24 11:19:45.555322 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-1b88c791-edcd-4336-a76e-eee2f9a48c84 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:45.557769 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-1b88c791-edcd-4336-a76e-eee2f9a48c84 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=788050c389224d17b163f24bb67902bf&device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 1096 time: 0.1426952 May 24 11:19:45.739075 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-720ba47e-791a-45f2-96ea-7ecade28ed04 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010 HTTP/1.1" status: 200 len: 217 time: 0.1684151 May 24 11:19:45.842665 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-f29cb982-9862-4d9a-9223-70aba0716642 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:45.844110 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-f29cb982-9862-4d9a-9223-70aba0716642 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=6a80c579-20d5-46c7-8c07-6438f1367afd HTTP/1.1" status: 200 len: 853 time: 0.0968237 May 24 11:19:45.857580 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5f8a4ac4-0690-41f8-b146-b274a80a05a0 req-df4b75b3-22af-460b-8bea-534a9bb809ae tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:45.864896 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5f8a4ac4-0690-41f8-b146-b274a80a05a0 req-df4b75b3-22af-460b-8bea-534a9bb809ae tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.2142394 May 24 11:19:45.967145 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5f8a4ac4-0690-41f8-b146-b274a80a05a0 req-b89a92cd-a02e-4fbb-8de2-952a29280c6c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1013341 May 24 11:19:45.995098 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-5d378899-4b67-4f22-a7d5-6451e674fea6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:45.997233 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-5d378899-4b67-4f22-a7d5-6451e674fea6 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1296380 May 24 11:19:46.183564 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9f96bc26-86c9-4d9a-8cd4-2c7703f654b5 req-9b4e5b19-3614-4308-ae71-7ead9f66f2dc tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:46.185883 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9f96bc26-86c9-4d9a-8cd4-2c7703f654b5 req-9b4e5b19-3614-4308-ae71-7ead9f66f2dc tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 945 time: 0.1212902 May 24 11:19:46.208329 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-74b2f355-ba1e-4efc-a249-7e0e6dcc440c service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2008531 May 24 11:19:46.258231 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9f96bc26-86c9-4d9a-8cd4-2c7703f654b5 req-d3a3c7e6-525c-47ea-87c6-42dcd1b26e94 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0651896 May 24 11:19:46.388699 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6394c79-8b9c-45b6-982a-93f74eadc4b2 req-47ce627d-d83f-483e-9a69-bf9df502c8ea service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1707723 May 24 11:19:46.596992 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-651d1fd2-8e40-4e00-9393-1effdd4cd4f0 req-eff81752-d4d2-434a-b9e3-1d2ff4f1b8dd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:46.598928 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-651d1fd2-8e40-4e00-9393-1effdd4cd4f0 req-eff81752-d4d2-434a-b9e3-1d2ff4f1b8dd service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=788050c389224d17b163f24bb67902bf&device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 1096 time: 0.1345851 May 24 11:19:46.609974 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5c550aaa-acc4-4be3-8e37-c6f8b57c1e5b req-b1b7c06d-31b2-46bb-ad49-045fa693690c tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:46.611881 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5c550aaa-acc4-4be3-8e37-c6f8b57c1e5b req-b1b7c06d-31b2-46bb-ad49-045fa693690c tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 943 time: 0.1959686 May 24 11:19:46.835441 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5c550aaa-acc4-4be3-8e37-c6f8b57c1e5b req-57f63dbd-45c0-4640-92c3-d452c3155b60 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1977177 May 24 11:19:46.889041 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-651d1fd2-8e40-4e00-9393-1effdd4cd4f0 req-549e6a08-05d7-469c-9923-d7360f32041b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:46.890815 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-651d1fd2-8e40-4e00-9393-1effdd4cd4f0 req-549e6a08-05d7-469c-9923-d7360f32041b service neutron] 10.222.0.93 "GET /v2.0/networks?id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 HTTP/1.1" status: 200 len: 915 time: 0.2779007 May 24 11:19:46.973278 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-651d1fd2-8e40-4e00-9393-1effdd4cd4f0 req-0b61b472-f70e-49fd-8610-ae22b399fae5 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010 HTTP/1.1" status: 200 len: 217 time: 0.0735228 May 24 11:19:47.077376 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-651d1fd2-8e40-4e00-9393-1effdd4cd4f0 req-2705c7db-a8ea-4c21-9863-5fab67c46485 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:47.079232 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-651d1fd2-8e40-4e00-9393-1effdd4cd4f0 req-2705c7db-a8ea-4c21-9863-5fab67c46485 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=6a80c579-20d5-46c7-8c07-6438f1367afd HTTP/1.1" status: 200 len: 853 time: 0.0987816 May 24 11:19:47.172032 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-651d1fd2-8e40-4e00-9393-1effdd4cd4f0 req-910a3e4a-0622-4138-a695-bbf02f755523 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:47.173555 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-651d1fd2-8e40-4e00-9393-1effdd4cd4f0 req-910a3e4a-0622-4138-a695-bbf02f755523 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0870342 May 24 11:19:47.224794 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c745c23c-2b70-4c25-bc4f-bf56b95de8eb req-61e92f81-8e10-41e3-9a2d-b4533ceea8c8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:47.233326 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c745c23c-2b70-4c25-bc4f-bf56b95de8eb req-61e92f81-8e10-41e3-9a2d-b4533ceea8c8 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1131332 May 24 11:19:47.288742 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c745c23c-2b70-4c25-bc4f-bf56b95de8eb req-037a8073-82b8-4a46-954a-9776f591371c tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0520256 May 24 11:19:47.682216 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-651d1fd2-8e40-4e00-9393-1effdd4cd4f0 req-058b49fa-62e3-4d97-bec5-0b9a952014e0 service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4922214 May 24 11:19:47.791503 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-50f6a8b5-5432-45de-aebd-1dac1d6d0ac8 req-cfd81b7d-650a-46b0-b853-115772a42f22 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:47.793673 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-50f6a8b5-5432-45de-aebd-1dac1d6d0ac8 req-cfd81b7d-650a-46b0-b853-115772a42f22 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 945 time: 0.2858486 May 24 11:19:47.880753 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-50f6a8b5-5432-45de-aebd-1dac1d6d0ac8 req-4a3cd40d-c3a0-4d5b-bce3-9d9671fa7fd5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0681026 May 24 11:19:47.942210 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-651d1fd2-8e40-4e00-9393-1effdd4cd4f0 req-5056c185-b4b1-46ed-8b70-37692c443094 service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2496519 May 24 11:19:48.248649 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c9c01170-7194-4060-80be-494914a915fa req-721a54ad-04e6-4437-91b6-b37befb086e4 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:48.250328 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c9c01170-7194-4060-80be-494914a915fa req-721a54ad-04e6-4437-91b6-b37befb086e4 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 945 time: 0.2942505 May 24 11:19:48.259879 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8573b02e-5595-473e-b246-f3e5924e6f8c req-85a35bb2-5185-43d8-b627-41d63085c7ba tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:48.267399 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8573b02e-5595-473e-b246-f3e5924e6f8c req-85a35bb2-5185-43d8-b627-41d63085c7ba tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 943 time: 0.2327287 May 24 11:19:48.532839 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eaa9962a-b360-4322-9d33-fdd855a2a03b req-05633cbf-5820-479e-985d-1442702d1207 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:48.534959 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-eaa9962a-b360-4322-9d33-fdd855a2a03b req-05633cbf-5820-479e-985d-1442702d1207 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1227124 May 24 11:19:48.589868 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-eaa9962a-b360-4322-9d33-fdd855a2a03b req-06d5963c-afd2-45b7-aa26-bb38d8254aff tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0485697 May 24 11:19:48.608355 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8573b02e-5595-473e-b246-f3e5924e6f8c req-7bbd8966-a479-4679-b53e-bbfcf5db1eb9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3397429 May 24 11:19:48.658068 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-c9c01170-7194-4060-80be-494914a915fa req-7c14a539-64ec-4b2f-999c-194c247731db tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Deleting port 159fe3b9-426c-472e-9870-2ead1792535a {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:19:48.658717 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-c9c01170-7194-4060-80be-494914a915fa req-7c14a539-64ec-4b2f-999c-194c247731db tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:48.821133 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-0a7eaa79-b28e-418a-bc95-ad7d286a25b7 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Synchronizing usage tracker for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:security_group {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:48.828914 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-c9c01170-7194-4060-80be-494914a915fa req-7c14a539-64ec-4b2f-999c-194c247731db tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] For port 159fe3b9-426c-472e-9870-2ead1792535a, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=159fe3b9-426c-472e-9870-2ead1792535a,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:48.829453 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-c9c01170-7194-4060-80be-494914a915fa req-7c14a539-64ec-4b2f-999c-194c247731db tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:48.841133 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-0a7eaa79-b28e-418a-bc95-ad7d286a25b7 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Unset dirty status for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:security_group {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:48.850019 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-0a7eaa79-b28e-418a-bc95-ad7d286a25b7 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-262467324&fields=id&tenant_id=9a8d61f79bbc496ca30e390fd9b5839e HTTP/1.1" status: 200 len: 267 time: 0.0739799 May 24 11:19:48.995329 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-c9c01170-7194-4060-80be-494914a915fa req-7c14a539-64ec-4b2f-999c-194c247731db tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Calling delete_port for 159fe3b9-426c-472e-9870-2ead1792535a owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:19:49.030023 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-f14cc022-2ad1-4001-9c8a-6097a4cf7431 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:49.033601 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-f14cc022-2ad1-4001-9c8a-6097a4cf7431 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 811 time: 0.1824024 May 24 11:19:49.058913 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-2b3aebfc-691c-4115-995f-49c60999d130 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/quotas/9a8d61f79bbc496ca30e390fd9b5839e HTTP/1.1" status: 200 len: 375 time: 0.0155895 May 24 11:19:49.124433 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8559553e-439f-42da-9a1b-89788790ca44 req-4200be6e-92fd-4a17-ae91-729db7dbcfc8 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:49.125837 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8559553e-439f-42da-9a1b-89788790ca44 req-4200be6e-92fd-4a17-ae91-729db7dbcfc8 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=d895da25-06b9-484c-8192-a6e4df06a856 HTTP/1.1" status: 200 len: 945 time: 0.1203225 May 24 11:19:49.204402 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-8649c291-0b0c-40e3-8680-0cec125239a0 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&fields=id HTTP/1.1" status: 200 len: 302 time: 0.1356926 May 24 11:19:49.254778 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device e0dbce52-01cb-4042-b340-ea0a6b7da010 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:19:49.315380 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8559553e-439f-42da-9a1b-89788790ca44 req-80a3fcba-07c3-4b9d-b8fe-6f225a260103 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1737554 May 24 11:19:49.336086 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-c9c01170-7194-4060-80be-494914a915fa req-7c14a539-64ec-4b2f-999c-194c247731db tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:49.357228 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-fdac78f2-de00-41b5-8091-2be92a4b3c51 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:49.419195 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7c14a539-64ec-4b2f-999c-194c247731db tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Pushing event deleted for resources: {'Port': ['ID=159fe3b9-426c-472e-9870-2ead1792535a,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:49.423383 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-7c14a539-64ec-4b2f-999c-194c247731db tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:49.491338 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port e0dbce52-01cb-4042-b340-ea0a6b7da010, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:49.494212 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-6c9c05e6-74b8-447e-827b-18b88ca753d1 service neutron] 10.222.0.18 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2740164 May 24 11:19:49.514248 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'port_id': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'mac_address': 'fa:16:3e:b3:e8:d3', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 539, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'ip_address': '172.20.0.6'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:19:49.518578 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['45911d6f-9595-40c5-8b29-75259c824bfe', '159fe3b9-426c-472e-9870-2ead1792535a'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:19:49.532829 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-c9c01170-7194-4060-80be-494914a915fa req-7c14a539-64ec-4b2f-999c-194c247731db tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] No DVR routers for this DVR port 159fe3b9-426c-472e-9870-2ead1792535a on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:19:49.536395 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-c9c01170-7194-4060-80be-494914a915fa req-7c14a539-64ec-4b2f-999c-194c247731db tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:49.539307 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'd895da25-06b9-484c-8192-a6e4df06a856', 'name': 'network-vif-deleted', 'tag': '159fe3b9-426c-472e-9870-2ead1792535a'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:19:49.568972 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c9c01170-7194-4060-80be-494914a915fa req-7c14a539-64ec-4b2f-999c-194c247731db tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "DELETE /v2.0/ports/159fe3b9-426c-472e-9870-2ead1792535a HTTP/1.1" status: 204 len: 173 time: 1.2985511 May 24 11:19:49.675031 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'd895da25-06b9-484c-8192-a6e4df06a856', 'name': 'network-vif-deleted', 'tag': '159fe3b9-426c-472e-9870-2ead1792535a', 'status': 'completed', 'code': 200} May 24 11:19:49.793453 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-e546df67-7daf-49e5-a705-ef00beb8b843 service neutron] 10.222.0.18 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2887058 May 24 11:19:49.828902 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:49.866408 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id e0dbce52-01cb-4042-b340-ea0a6b7da010 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:19:49.902686 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port e0dbce52-01cb-4042-b340-ea0a6b7da010, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:49.918725 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:49.926787 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b5d72999-3905-4a77-818c-efb69e4010a8 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:49.967551 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-62b9ce08-f810-4f21-9a7e-50115c11fed2 req-d25e536d-088c-4fbc-8d53-8886d3d7c82a tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:49.976343 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-62b9ce08-f810-4f21-9a7e-50115c11fed2 req-d25e536d-088c-4fbc-8d53-8886d3d7c82a tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1349001 May 24 11:19:50.015541 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=e0dbce52-01cb-4042-b340-ea0a6b7da010,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:50.024962 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:50.025665 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:50.026369 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:50.026733 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:50.027076 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:50.027413 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:50.027767 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:50.028676 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:50.029019 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:50.029387 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:50.029766 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:50.030108 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:19:50.034505 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:50.042709 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-62b9ce08-f810-4f21-9a7e-50115c11fed2 req-85815876-cd6d-43ff-96c9-492469304ccd tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0645065 May 24 11:19:50.242085 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['e0dbce52-01cb-4042-b340-ea0a6b7da010'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:19:50.304517 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-abae0d3f-2a64-40e8-b096-c86a0ab44d05 req-e8d26218-0a55-49aa-b95a-f26fe3ac00a6 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:50.322079 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-abae0d3f-2a64-40e8-b096-c86a0ab44d05 req-e8d26218-0a55-49aa-b95a-f26fe3ac00a6 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 944 time: 0.1569765 May 24 11:19:50.459457 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-abae0d3f-2a64-40e8-b096-c86a0ab44d05 req-04f38c31-a8fe-419a-9730-fee145efee20 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1355407 May 24 11:19:50.614671 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-05c4c368-f92b-482a-8dbe-909eb1860d42 req-27631429-3383-4f48-a8f6-861ecc9f0de5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Synchronizing usage tracker for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:50.653758 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-05c4c368-f92b-482a-8dbe-909eb1860d42 req-27631429-3383-4f48-a8f6-861ecc9f0de5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Unset dirty status for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:50.664934 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-05c4c368-f92b-482a-8dbe-909eb1860d42 req-27631429-3383-4f48-a8f6-861ecc9f0de5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 210 time: 0.1265216 May 24 11:19:51.285140 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device e0dbce52-01cb-4042-b340-ea0a6b7da010 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:19:51.343009 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d49030a-0312-4ddc-b017-713cbcd90904 req-aa3938f8-c3bf-41ff-b965-6d685c3c03f3 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:51.345416 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6d49030a-0312-4ddc-b017-713cbcd90904 req-aa3938f8-c3bf-41ff-b965-6d685c3c03f3 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1641715 May 24 11:19:51.435014 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6d49030a-0312-4ddc-b017-713cbcd90904 req-33b78acc-547c-4f4b-a84a-4879b92aaa0b tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0785642 May 24 11:19:51.471044 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port e0dbce52-01cb-4042-b340-ea0a6b7da010 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:19:51.477014 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port e0dbce52-01cb-4042-b340-ea0a6b7da010 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:51.477783 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:51.633555 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:51.669733 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:19:51.698766 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port e0dbce52-01cb-4042-b340-ea0a6b7da010, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:51.706990 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:51.708319 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 159fe3b9-426c-472e-9870-2ead1792535a no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:19:51.809284 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'code': 200} May 24 11:19:51.810378 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-1ff26346-8675-43df-b264-74dc1c913b76 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:51.895138 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38f0ec67-c059-4d6a-9905-3a604836ce33 req-d8ee3dc6-a3c2-47eb-bae8-76b653096efe tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:51.898801 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-38f0ec67-c059-4d6a-9905-3a604836ce33 req-d8ee3dc6-a3c2-47eb-bae8-76b653096efe tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 945 time: 0.2139387 May 24 11:19:51.975226 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 159fe3b9-426c-472e-9870-2ead1792535a {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:19:51.975816 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 159fe3b9-426c-472e-9870-2ead1792535a not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:19:52.052564 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-38f0ec67-c059-4d6a-9905-3a604836ce33 req-24a1e6e6-746a-47b5-b620-4b3b9826fca2 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1438270 May 24 11:19:52.092385 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:52.110969 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=e0dbce52-01cb-4042-b340-ea0a6b7da010,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:52.122635 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.312s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:52.130300 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-96a01e89-5d86-4c62-ab7d-307305fdf665 req-eda4a0e6-4a58-4b08-b043-5601190fbc43 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:52.136440 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-96a01e89-5d86-4c62-ab7d-307305fdf665 req-eda4a0e6-4a58-4b08-b043-5601190fbc43 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.93 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.1824479 May 24 11:19:52.342735 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-912e4eda-7c24-48da-8cb3-0015a72c7434 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:52.344286 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-912e4eda-7c24-48da-8cb3-0015a72c7434 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 811 time: 0.3763373 May 24 11:19:52.415316 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-db38f73b-7429-4dfd-a31e-7688218815de req-154c541e-b956-4346-8749-37a413a60fad tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 210 time: 0.2352061 May 24 11:19:52.453178 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-ebfeac0b-1a10-412b-a0a0-681656110db8 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=9a8d61f79bbc496ca30e390fd9b5839e HTTP/1.1" status: 200 len: 379 time: 0.0883634 May 24 11:19:52.482258 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'port': {'device_id': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'admin_state_up': True, 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'security_groups': ['5928fed2-3207-44a0-91e4-485200bd1a27']}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:52.515886 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd828c2c-5157-485f-8240-52caed9508de req-990a7625-91a7-4ab5-9053-2e4c740aae94 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:52.517670 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fd828c2c-5157-485f-8240-52caed9508de req-990a7625-91a7-4ab5-9053-2e4c740aae94 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 1098 time: 0.2467141 May 24 11:19:52.587420 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fd828c2c-5157-485f-8240-52caed9508de req-9a56d6f7-3da5-4333-a574-4a38ab771841 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0561345 May 24 11:19:52.594039 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-96a01e89-5d86-4c62-ab7d-307305fdf665 req-b7f4c6d4-a057-4298-8e30-3050703dfda9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Deleting port 02210ade-8c60-41b6-aa82-a104d4fc0447 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:19:52.594631 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-96a01e89-5d86-4c62-ab7d-307305fdf665 req-b7f4c6d4-a057-4298-8e30-3050703dfda9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:52.761952 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-96a01e89-5d86-4c62-ab7d-307305fdf665 req-b7f4c6d4-a057-4298-8e30-3050703dfda9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] For port 02210ade-8c60-41b6-aa82-a104d4fc0447, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=02210ade-8c60-41b6-aa82-a104d4fc0447,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:52.762438 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-96a01e89-5d86-4c62-ab7d-307305fdf665 req-b7f4c6d4-a057-4298-8e30-3050703dfda9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:52.895037 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:19:52.914037 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:19:52.942399 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Made reservation on behalf of 9a8d61f79bbc496ca30e390fd9b5839e for: {'port': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:19:52.947750 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:52.973914 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d014d30c-9269-41ed-ab49-c8b11daa24f6 req-03963391-7b83-40a7-abd0-7be2df00d753 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:52.975461 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d014d30c-9269-41ed-ab49-c8b11daa24f6 req-03963391-7b83-40a7-abd0-7be2df00d753 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 946 time: 0.2113864 May 24 11:19:53.079158 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-96a01e89-5d86-4c62-ab7d-307305fdf665 req-b7f4c6d4-a057-4298-8e30-3050703dfda9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Calling delete_port for 02210ade-8c60-41b6-aa82-a104d4fc0447 owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:19:53.167549 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b376579a-5e9d-4528-a55f-b06c8dd768ca req-85f06e93-c7b0-497e-bfa0-3e464aff057b tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:53.171402 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-b376579a-5e9d-4528-a55f-b06c8dd768ca req-85f06e93-c7b0-497e-bfa0-3e464aff057b tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Synchronizing usage tracker for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:53.208412 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d014d30c-9269-41ed-ab49-c8b11daa24f6 req-f9c74f89-bda3-411d-97b7-c1ef23be1612 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/security-groups?id=cbdda050-525d-4636-8079-44cfb0e04b03&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2266324 May 24 11:19:53.244193 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-b376579a-5e9d-4528-a55f-b06c8dd768ca req-85f06e93-c7b0-497e-bfa0-3e464aff057b tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Unset dirty status for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:53.253081 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b376579a-5e9d-4528-a55f-b06c8dd768ca req-85f06e93-c7b0-497e-bfa0-3e464aff057b tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 945 time: 0.4788003 May 24 11:19:53.414671 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-96a01e89-5d86-4c62-ab7d-307305fdf665 req-b7f4c6d4-a057-4298-8e30-3050703dfda9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:53.440628 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:53.507536 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8bc3253a-71f7-469e-a51e-26fb60b5d723 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:53.575367 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Allocated IP 172.20.0.5 (be65d1af-351e-4e5b-8c19-1b04ff5cfc29/994391be-baae-4f60-9ef9-bf02d19d3477/8a7c6451-6355-44ad-a36a-f02ffcfe252c) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:19:53.599597 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b7f4c6d4-a057-4298-8e30-3050703dfda9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Pushing event deleted for resources: {'Port': ['ID=02210ade-8c60-41b6-aa82-a104d4fc0447,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:53.626003 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b7f4c6d4-a057-4298-8e30-3050703dfda9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.185s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:53.679070 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b376579a-5e9d-4528-a55f-b06c8dd768ca req-8aded34d-3cef-4a6b-8bb7-129074ec4e79 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.4219866 May 24 11:19:53.751414 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-96a01e89-5d86-4c62-ab7d-307305fdf665 req-b7f4c6d4-a057-4298-8e30-3050703dfda9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] No DVR routers for this DVR port 02210ade-8c60-41b6-aa82-a104d4fc0447 on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:19:53.754526 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-96a01e89-5d86-4c62-ab7d-307305fdf665 req-b7f4c6d4-a057-4298-8e30-3050703dfda9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:53.761923 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'd9e36d1d-343c-4b97-b226-9b58722aebb2', 'name': 'network-vif-deleted', 'tag': '02210ade-8c60-41b6-aa82-a104d4fc0447'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:19:53.782277 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-96a01e89-5d86-4c62-ab7d-307305fdf665 req-b7f4c6d4-a057-4298-8e30-3050703dfda9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.93 "DELETE /v2.0/ports/02210ade-8c60-41b6-aa82-a104d4fc0447 HTTP/1.1" status: 204 len: 173 time: 1.6383834 May 24 11:19:53.866118 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-332474ee-faef-49f5-a392-9449ef94fce1 req-691d0de7-47cc-405e-b746-c8e849ff4f42 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:53.870374 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-332474ee-faef-49f5-a392-9449ef94fce1 req-691d0de7-47cc-405e-b746-c8e849ff4f42 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 945 time: 0.2784173 May 24 11:19:53.939153 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:53.943725 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e56bfc85-70e0-4d9c-a6e0-61d947a27595 req-5f696bc2-5f11-4a98-b97a-50b830a5b219 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 210 time: 0.1111686 May 24 11:19:53.976417 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-332474ee-faef-49f5-a392-9449ef94fce1 req-351c6c11-748b-4d0f-9dbe-e512ec7077cb tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0892909 May 24 11:19:53.989975 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Transition to ACTIVE for port object 8a7c6451-6355-44ad-a36a-f02ffcfe252c will not be triggered until provisioned by entity DHCP. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:19:54.042436 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:54.053393 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:54.062126 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'd9e36d1d-343c-4b97-b226-9b58722aebb2', 'name': 'network-vif-deleted', 'tag': '02210ade-8c60-41b6-aa82-a104d4fc0447', 'status': 'completed', 'code': 200} May 24 11:19:54.142483 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:19:54.147047 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:19:54.151608 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:19:54.208060 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'Port': ['ID=8a7c6451-6355-44ad-a36a-f02ffcfe252c,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:54.214188 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.163s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:54.248343 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:54.249013 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:54.285684 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-257ac2c0-479d-41ad-b5db-1fd7b52eeff4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.8161497 May 24 11:19:54.305627 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Request body: {'port': {'device_id': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:19:54.528273 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-95f296e3-1eb6-4441-a446-c72ee053794f None None] Provisioning for port 8a7c6451-6355-44ad-a36a-f02ffcfe252c completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:19:54.533613 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:54.537839 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-95f296e3-1eb6-4441-a446-c72ee053794f None None] Provisioning complete for port 8a7c6451-6355-44ad-a36a-f02ffcfe252c triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:19:54.540565 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-95f296e3-1eb6-4441-a446-c72ee053794f None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:54.568050 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-8a251dd0-6037-4374-8c08-585a6d8277a7 req-7f42cf91-e618-474a-8648-8e4934f3d92a tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Synchronizing usage tracker for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:54.600490 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-8a251dd0-6037-4374-8c08-585a6d8277a7 req-7f42cf91-e618-474a-8648-8e4934f3d92a tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Unset dirty status for tenant:bf5fa11a013d4e1a8b86edc5fe49e97b on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:54.601761 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8a251dd0-6037-4374-8c08-585a6d8277a7 req-7f42cf91-e618-474a-8648-8e4934f3d92a tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] 10.222.0.18 "GET /v2.0/ports?device_id=d9e36d1d-343c-4b97-b226-9b58722aebb2 HTTP/1.1" status: 200 len: 210 time: 0.1385036 May 24 11:19:54.626715 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-33815a45-8872-4c2c-a8ad-2e6d98b6793f req-4a73d53c-58e0-464b-83d1-63131bcb69a4 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:54.630081 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-33815a45-8872-4c2c-a8ad-2e6d98b6793f req-4a73d53c-58e0-464b-83d1-63131bcb69a4 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 945 time: 0.2517183 May 24 11:19:54.667186 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-95f296e3-1eb6-4441-a446-c72ee053794f None None] Port 8a7c6451-6355-44ad-a36a-f02ffcfe252c cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:19:54.848664 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-33815a45-8872-4c2c-a8ad-2e6d98b6793f req-bc9fe0d9-c749-4ba5-9b49-994a23c67963 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Deleting port 45911d6f-9595-40c5-8b29-75259c824bfe {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:19:54.849006 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-33815a45-8872-4c2c-a8ad-2e6d98b6793f req-bc9fe0d9-c749-4ba5-9b49-994a23c67963 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:55.046303 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4eec1a78-c16b-4db7-81d9-8fa207cabbaa req-818ff7c5-db52-4b19-98ee-68feaba54db7 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:55.053879 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4eec1a78-c16b-4db7-81d9-8fa207cabbaa req-818ff7c5-db52-4b19-98ee-68feaba54db7 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 945 time: 0.2436941 May 24 11:19:55.067218 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-33815a45-8872-4c2c-a8ad-2e6d98b6793f req-bc9fe0d9-c749-4ba5-9b49-994a23c67963 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] For port 45911d6f-9595-40c5-8b29-75259c824bfe, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=45911d6f-9595-40c5-8b29-75259c824bfe,segment=NetworkSegment(b756bc07-d062-4b63-a043-c614b00fa2ca),segment_id=b756bc07-d062-4b63-a043-c614b00fa2ca)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:55.068103 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-33815a45-8872-4c2c-a8ad-2e6d98b6793f req-bc9fe0d9-c749-4ba5-9b49-994a23c67963 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:55.164735 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4eec1a78-c16b-4db7-81d9-8fa207cabbaa req-da37ae5c-149b-40aa-831b-00797de27a1d tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/security-groups?id=b9070542-7975-4acb-bb57-8c4a7c17e6fc&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1098981 May 24 11:19:55.185586 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 8a7c6451-6355-44ad-a36a-f02ffcfe252c {{(pid=91820) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:19:55.194766 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:55.212412 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Transition to ACTIVE for port object 8a7c6451-6355-44ad-a36a-f02ffcfe252c will not be triggered until provisioned by entity L2. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:19:55.219067 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:55.224014 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-8bfbea10-d1b8-420b-86e6-597159689b46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:55.243941 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-33815a45-8872-4c2c-a8ad-2e6d98b6793f req-bc9fe0d9-c749-4ba5-9b49-994a23c67963 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Calling delete_port for 45911d6f-9595-40c5-8b29-75259c824bfe owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:19:55.323691 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 45911d6f-9595-40c5-8b29-75259c824bfe no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:19:55.425651 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aa6d2a08-8251-4fc3-bc51-62b04bb618b7 req-0066d662-1bbe-4796-b96c-d9aa88b39405 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:55.438607 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-aa6d2a08-8251-4fc3-bc51-62b04bb618b7 req-0066d662-1bbe-4796-b96c-d9aa88b39405 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 943 time: 0.1836219 May 24 11:19:55.497025 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:55.532004 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Pushing event updated for resources: {'Port': ['ID=8a7c6451-6355-44ad-a36a-f02ffcfe252c,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:55.539589 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.315s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:55.611080 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-aa6d2a08-8251-4fc3-bc51-62b04bb618b7 req-8867b705-369e-419b-8a6d-55da232edf3a tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Synchronizing usage tracker for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:55.641976 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-aa6d2a08-8251-4fc3-bc51-62b04bb618b7 req-8867b705-369e-419b-8a6d-55da232edf3a tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Unset dirty status for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:55.643160 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-aa6d2a08-8251-4fc3-bc51-62b04bb618b7 req-8867b705-369e-419b-8a6d-55da232edf3a tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2038522 May 24 11:19:55.649133 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-33815a45-8872-4c2c-a8ad-2e6d98b6793f req-bc9fe0d9-c749-4ba5-9b49-994a23c67963 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:55.651472 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-20ae2225-deb9-4dd3-a401-ec7c395344ca req-d5f9ceb5-3aa4-4afc-ac73-913c66cb973b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:55.653161 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-20ae2225-deb9-4dd3-a401-ec7c395344ca req-d5f9ceb5-3aa4-4afc-ac73-913c66cb973b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 945 time: 0.3499088 May 24 11:19:55.692389 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b7f4c6d4-a057-4298-8e30-3050703dfda9 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:55.712186 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 02210ade-8c60-41b6-aa82-a104d4fc0447 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:19:55.742714 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bc9fe0d9-c749-4ba5-9b49-994a23c67963 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Pushing event deleted for resources: {'Port': ['ID=45911d6f-9595-40c5-8b29-75259c824bfe,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:55.747881 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-20ae2225-deb9-4dd3-a401-ec7c395344ca req-2f7806c4-fb9c-4ac2-b099-d8f658f36a0e tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0786674 May 24 11:19:55.759652 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Attempting to bind port 8a7c6451-6355-44ad-a36a-f02ffcfe252c on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:19:55.760139 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Attempting to bind port 8a7c6451-6355-44ad-a36a-f02ffcfe252c by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:19:55.760563 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Attempting to bind port 8a7c6451-6355-44ad-a36a-f02ffcfe252c on network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:19:55.781017 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Port 8a7c6451-6355-44ad-a36a-f02ffcfe252c on network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:19:55.781516 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Attempting to bind port 8a7c6451-6355-44ad-a36a-f02ffcfe252c on network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:19:55.789077 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 02210ade-8c60-41b6-aa82-a104d4fc0447 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:19:55.789296 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 02210ade-8c60-41b6-aa82-a104d4fc0447 not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:19:55.790056 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 19, 52), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:19:55.790714 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Checking segment: {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:19:55.791424 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Bound using segment: {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:19:55.793208 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Bound port: 8a7c6451-6355-44ad-a36a-f02ffcfe252c, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'}}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:19:55.793208 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:55.793907 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:55.821218 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-bc9fe0d9-c749-4ba5-9b49-994a23c67963 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:55.874002 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 57140) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:55.895551 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-83f8fe30-ae49-431d-bf08-183dfa3eca4a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:55.918282 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-33815a45-8872-4c2c-a8ad-2e6d98b6793f req-bc9fe0d9-c749-4ba5-9b49-994a23c67963 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] No DVR routers for this DVR port 45911d6f-9595-40c5-8b29-75259c824bfe on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:19:55.921973 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-33815a45-8872-4c2c-a8ad-2e6d98b6793f req-bc9fe0d9-c749-4ba5-9b49-994a23c67963 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:55.939134 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-33815a45-8872-4c2c-a8ad-2e6d98b6793f req-bc9fe0d9-c749-4ba5-9b49-994a23c67963 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.93 "DELETE /v2.0/ports/45911d6f-9595-40c5-8b29-75259c824bfe HTTP/1.1" status: 204 len: 173 time: 1.3022394 May 24 11:19:56.063505 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'fd3abcad-fe0f-460b-b250-becb3ccb2624', 'name': 'network-vif-deleted', 'tag': '45911d6f-9595-40c5-8b29-75259c824bfe'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:19:56.132972 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'fd3abcad-fe0f-460b-b250-becb3ccb2624', 'name': 'network-vif-deleted', 'tag': '45911d6f-9595-40c5-8b29-75259c824bfe', 'status': 'completed', 'code': 200} May 24 11:19:56.330787 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] For port 8a7c6451-6355-44ad-a36a-f02ffcfe252c, host hv-cinder-78409, cleared binding levels {{(pid=91820) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:19:56.338139 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] For port 8a7c6451-6355-44ad-a36a-f02ffcfe252c, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=8a7c6451-6355-44ad-a36a-f02ffcfe252c,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91820) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:19:56.365890 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:56.374553 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:56.382247 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-46b218ad-51bf-406c-8ec0-0b674abd4c69 req-83455fc0-fa86-4515-90e8-c589fb84a1b9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Synchronizing usage tracker for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:19:56.408011 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-46b218ad-51bf-406c-8ec0-0b674abd4c69 req-83455fc0-fa86-4515-90e8-c589fb84a1b9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Unset dirty status for tenant:003eb49c73ae4e6a962cbfb425b3c1eb on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:19:56.409297 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-46b218ad-51bf-406c-8ec0-0b674abd4c69 req-83455fc0-fa86-4515-90e8-c589fb84a1b9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] 10.222.0.18 "GET /v2.0/ports?device_id=fd3abcad-fe0f-460b-b250-becb3ccb2624 HTTP/1.1" status: 200 len: 210 time: 0.0861387 May 24 11:19:56.464153 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a4c5b737-7314-43b7-83ae-a6fa644a58a8 req-2a786e59-07fe-430e-9358-5b3d29a75714 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:56.467460 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a4c5b737-7314-43b7-83ae-a6fa644a58a8 req-2a786e59-07fe-430e-9358-5b3d29a75714 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 945 time: 0.4058342 May 24 11:19:56.574425 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:56.574930 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:56.576366 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] 10.222.0.93 "PUT /v2.0/ports/8a7c6451-6355-44ad-a36a-f02ffcfe252c HTTP/1.1" status: 200 len: 1093 time: 2.2760229 May 24 11:19:56.577202 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'tag': '8a7c6451-6355-44ad-a36a-f02ffcfe252c'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:19:56.753013 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'tag': '8a7c6451-6355-44ad-a36a-f02ffcfe252c', 'status': 'completed', 'code': 200} May 24 11:19:56.872265 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a4c5b737-7314-43b7-83ae-a6fa644a58a8 req-8d8358f9-9033-46d2-afbb-269d4be086a8 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3056076 May 24 11:19:56.883923 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Pushing event updated for resources: {'Port': ['ID=8a7c6451-6355-44ad-a36a-f02ffcfe252c,revision_number=3']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:56.889923 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.515s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:56.938056 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0bdaa566-54da-4e0d-8551-0e75bd8d4439 req-393d7c7e-1132-4b0b-8bc9-d8147dc971a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:56.958052 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0bdaa566-54da-4e0d-8551-0e75bd8d4439 req-393d7c7e-1132-4b0b-8bc9-d8147dc971a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 943 time: 0.1416848 May 24 11:19:56.978631 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:57.013523 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-14ab0b4b-7e66-472b-910c-5654a1b934b5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:57.015830 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-14ab0b4b-7e66-472b-910c-5654a1b934b5 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 1096 time: 0.3076353 May 24 11:19:57.037006 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0bdaa566-54da-4e0d-8551-0e75bd8d4439 req-10acc685-2e92-4684-8960-1e0f909122e8 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0777991 May 24 11:19:57.154331 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-c347afb5-a6c3-467c-b8c6-baf92ed303d0 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=8a7c6451-6355-44ad-a36a-f02ffcfe252c HTTP/1.1" status: 200 len: 217 time: 0.1305459 May 24 11:19:57.297122 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-4d833ec8-ff7a-4005-adbe-c0baac442cae service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:57.298565 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-4d833ec8-ff7a-4005-adbe-c0baac442cae service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.0908794 May 24 11:19:57.424045 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-894463e9-c35e-4543-9577-0a457a6b6114 req-a657ab5a-2fe4-4c16-a655-107150f20a36 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:57.432092 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-894463e9-c35e-4543-9577-0a457a6b6114 req-a657ab5a-2fe4-4c16-a655-107150f20a36 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 945 time: 0.1095040 May 24 11:19:57.466307 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-57c3a4c2-65b8-46bf-9c94-c45384d4a5a9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:57.468828 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-57c3a4c2-65b8-46bf-9c94-c45384d4a5a9 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1575167 May 24 11:19:57.490574 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-894463e9-c35e-4543-9577-0a457a6b6114 req-52f61ca6-e74d-42b2-bbbb-439635af3965 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0574007 May 24 11:19:57.540614 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57168) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:57.635180 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-687465'] for router_interface, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:57.760037 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_v2 [None req-d7c73e72-ecc7-4470-9f8a-436b3fa3194c admin admin] Deleting subnet 1107b450-6c27-4f13-8b6e-abdbbbb1ea0f {{(pid=91819) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:19:57.770715 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d7c73e72-ecc7-4470-9f8a-436b3fa3194c admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:57.851580 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-f22a8f10-4976-452e-ab01-40b542cecbda service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3737586 May 24 11:19:57.866170 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d7c73e72-ecc7-4470-9f8a-436b3fa3194c admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534853172'] for subnet, precommit_delete_associations {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:57.959248 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d7c73e72-ecc7-4470-9f8a-436b3fa3194c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:58.313220 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-76a73dda-c30a-41e4-8792-29296cb4f9f5 req-1fb3b015-5886-400d-87ba-b5ed606619a7 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4540036 May 24 11:19:58.528782 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-41570fe3-d8df-4c83-9ebc-decf67c3f79f req-0dfeff99-aaa8-4ca3-af1b-c03aff4ab54d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:58.530481 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-41570fe3-d8df-4c83-9ebc-decf67c3f79f req-0dfeff99-aaa8-4ca3-af1b-c03aff4ab54d service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 1096 time: 0.1569407 May 24 11:19:58.615911 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63b76385-bb40-4894-9a16-63560c17115f req-ae3548f2-3a90-4b92-87b6-dbeee5de9d73 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:58.625697 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-63b76385-bb40-4894-9a16-63560c17115f req-ae3548f2-3a90-4b92-87b6-dbeee5de9d73 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 943 time: 0.2022569 May 24 11:19:58.664043 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d7c73e72-ecc7-4470-9f8a-436b3fa3194c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534944900', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535570805'] for subnet, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:58.694807 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:58.743523 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] Deleting port 97fb6682-0049-400a-84fe-731434f40cc6 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:19:58.743871 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:58.751639 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-63b76385-bb40-4894-9a16-63560c17115f req-c53c41a7-b90f-43e7-8a05-8337d00b1be5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1243525 May 24 11:19:58.751639 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d7c73e72-ecc7-4470-9f8a-436b3fa3194c admin admin] Pushing event deleted for resources: {'Subnet': ['ID=1107b450-6c27-4f13-8b6e-abdbbbb1ea0f,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:58.754015 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d7c73e72-ecc7-4470-9f8a-436b3fa3194c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:58.757434 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-41570fe3-d8df-4c83-9ebc-decf67c3f79f req-7fdc33e3-c7c3-4397-a6a5-3f9d9131b44b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:58.757568 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d7c73e72-ecc7-4470-9f8a-436b3fa3194c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:58.759352 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-41570fe3-d8df-4c83-9ebc-decf67c3f79f req-7fdc33e3-c7c3-4397-a6a5-3f9d9131b44b service neutron] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 912 time: 0.1909573 May 24 11:19:58.765449 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-d7c73e72-ecc7-4470-9f8a-436b3fa3194c admin admin] 10.222.0.18 "DELETE /v2.0/subnets/1107b450-6c27-4f13-8b6e-abdbbbb1ea0f HTTP/1.1" status: 204 len: 168 time: 1.2234156 May 24 11:19:58.768110 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57188) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:58.888822 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-41570fe3-d8df-4c83-9ebc-decf67c3f79f req-9f4b5a73-988e-4fd6-b09f-c9f54bac6508 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=8a7c6451-6355-44ad-a36a-f02ffcfe252c HTTP/1.1" status: 200 len: 217 time: 0.0814633 May 24 11:19:58.919192 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:58.933344 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:59.001634 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-41570fe3-d8df-4c83-9ebc-decf67c3f79f req-f8f80b3c-3553-4d4e-a927-e0e3f87f5186 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:59.003134 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-41570fe3-d8df-4c83-9ebc-decf67c3f79f req-f8f80b3c-3553-4d4e-a927-e0e3f87f5186 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.0953414 May 24 11:19:59.072400 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] For port 97fb6682-0049-400a-84fe-731434f40cc6, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=97fb6682-0049-400a-84fe-731434f40cc6,segment=NetworkSegment(6bc6a763-352e-4ddf-bea5-3721ee687b85),segment_id=6bc6a763-352e-4ddf-bea5-3721ee687b85)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:19:59.073634 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:59.077582 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:59.096407 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['bb3e0f6d-2046-4828-8f43-5f99ba5f3add']}} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:59.112836 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'b756bc07-d062-4b63-a043-c614b00fa2ca') {'for_net_delete': True} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:59.113184 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'b756bc07-d062-4b63-a043-c614b00fa2ca') {} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:19:59.125885 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534853183'] for segment, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:59.150778 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:59.160453 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] Releasing vlan 524 on physical network physnet1 to pool {{(pid=91819) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:19:59.160943 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535155490'] for segment, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: Traceback (most recent call last): May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: timer() May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: cb(*args, **kw) May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: return func(*args, **kwargs) May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: self._notify() May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: self.callback(batched_events) May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: event.method(event) May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: return f(self, *a, **k) May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._get(url).json() May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: return self.request(url, 'GET', **kwargs) May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:19:59.174763 dv-cinder-784092-9-iscsi neutron-server[91819]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:19:59.225754 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] Calling delete_port for 97fb6682-0049-400a-84fe-731434f40cc6 owned by network:router_interface {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:19:59.270112 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-41570fe3-d8df-4c83-9ebc-decf67c3f79f req-e6ab4a0c-dc16-42c1-93ca-97757a6a41b6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:19:59.271627 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-41570fe3-d8df-4c83-9ebc-decf67c3f79f req-e6ab4a0c-dc16-42c1-93ca-97757a6a41b6 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.2587039 May 24 11:19:59.286189 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:59.295518 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:19:59.305588 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:19:59.332448 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] Pushing event deleted for resources: {'Network': ['ID=bb3e0f6d-2046-4828-8f43-5f99ba5f3add,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:19:59.339611 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:19:59.344979 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-4a69165b-b522-408b-b991-808c07a63a6c admin admin] 10.222.0.18 "DELETE /v2.0/networks/bb3e0f6d-2046-4828-8f43-5f99ba5f3add HTTP/1.1" status: 204 len: 168 time: 0.5757210 May 24 11:19:59.674034 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-41570fe3-d8df-4c83-9ebc-decf67c3f79f req-3a596739-7456-41ac-a2f3-a6114530c555 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3954000 May 24 11:19:59.714559 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57202) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:59.833146 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-3b7f42de-a5e1-448c-b787-5672367c7731 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=003eb49c73ae4e6a962cbfb425b3c1eb&name=default HTTP/1.1" status: 200 len: 2975 time: 0.1151614 May 24 11:19:59.834612 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57210) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:19:59.979908 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-886b88a8-2f76-4190-837d-118df2ae4b33 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:00.031172 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-886b88a8-2f76-4190-837d-118df2ae4b33 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:00.083616 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:00.102445 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-78b94553-a416-4043-91df-d8ddb1bdc4bb tempest-DeleteServersTestJSON-919450171 tempest-DeleteServersTestJSON-919450171-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:00.131867 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-886b88a8-2f76-4190-837d-118df2ae4b33 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:00.146113 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-597dc802-9edc-4b32-ad82-fb2134f5e068 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:00.163078 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-41570fe3-d8df-4c83-9ebc-decf67c3f79f req-bad16274-972e-431a-9954-b51f51f2e87f service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4809549 May 24 11:20:00.168442 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-886b88a8-2f76-4190-837d-118df2ae4b33 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b9070542-7975-4acb-bb57-8c4a7c17e6fc,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:00.184415 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-886b88a8-2f76-4190-837d-118df2ae4b33 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:00.195411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-886b88a8-2f76-4190-837d-118df2ae4b33 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:00.196683 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-886b88a8-2f76-4190-837d-118df2ae4b33 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/b9070542-7975-4acb-bb57-8c4a7c17e6fc HTTP/1.1" status: 204 len: 168 time: 0.3597274 May 24 11:20:00.229263 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] Pushing event deleted for resources: {'Port': ['ID=97fb6682-0049-400a-84fe-731434f40cc6,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:00.241448 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.139s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:00.296484 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9c975729-61ed-4bdb-99f5-f635879e89ad req-dbbd06c9-6b52-4b78-825a-c8afb52d9c72 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:00.297578 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-9c975729-61ed-4bdb-99f5-f635879e89ad req-dbbd06c9-6b52-4b78-825a-c8afb52d9c72 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Synchronizing usage tracker for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:20:00.350269 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-9c975729-61ed-4bdb-99f5-f635879e89ad req-dbbd06c9-6b52-4b78-825a-c8afb52d9c72 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Unset dirty status for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:20:00.366743 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9c975729-61ed-4bdb-99f5-f635879e89ad req-dbbd06c9-6b52-4b78-825a-c8afb52d9c72 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 943 time: 0.4068258 May 24 11:20:00.515445 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9c975729-61ed-4bdb-99f5-f635879e89ad req-c11e81e4-35b2-420e-ba47-7ee2d0d9cfc6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1475940 May 24 11:20:00.747032 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-537767', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-685561'] for router_interface, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:01.293360 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] Router 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:20:01.303041 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:20:01.519426 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:20:01.528558 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1baa03a2-b534-432e-94fe-7ddc4a617b54 None None] Sync routers for ids ['24e49d59-ce93-4246-ae1a-cf4080a4b8f5'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:20:01.548974 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1baa03a2-b534-432e-94fe-7ddc4a617b54 None None] Routers {'24e49d59-ce93-4246-ae1a-cf4080a4b8f5'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:20:01.556727 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-1baa03a2-b534-432e-94fe-7ddc4a617b54 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:19:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['24e49d59-ce93-4246-ae1a-cf4080a4b8f5'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:01.568727 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] 10.222.0.18 "PUT /v2.0/routers/24e49d59-ce93-4246-ae1a-cf4080a4b8f5/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 5.6890602 May 24 11:20:01.571471 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 57258) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:01.916127 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-1baa03a2-b534-432e-94fe-7ddc4a617b54 None None] Set router 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:20:01.917731 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Notify callbacks [] for router, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:02.198442 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9451b847-d597-4f07-afa8-32c3d2214c8d req-27a11fc7-f829-49a5-8749-c5ebd6f546aa tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:02.200180 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9451b847-d597-4f07-afa8-32c3d2214c8d req-27a11fc7-f829-49a5-8749-c5ebd6f546aa tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 943 time: 0.4165826 May 24 11:20:02.289640 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9451b847-d597-4f07-afa8-32c3d2214c8d req-d7e4153c-f281-444a-ade5-800944694a45 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0817502 May 24 11:20:02.322647 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-1baa03a2-b534-432e-94fe-7ddc4a617b54 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'name': 'tempest-ServerRescueNegativeTestJSON-12783889-router', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233'}], 'enable_snat': True}, 'gw_port_id': 'c9840383-de18-463c-8858-5536335f4466', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:16:04Z', 'updated_at': '2021-05-24T11:19:59Z', 'revision_number': 5, 'project_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'gw_port': {'id': 'c9840383-de18-463c-8858-5536335f4466', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:07:5a:29', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 81, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.233', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:06Z', 'updated_at': '2021-05-24T11:16:19Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:02.329648 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1baa03a2-b534-432e-94fe-7ddc4a617b54 None None] Checking router: 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:20:02.336912 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:02.344041 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1baa03a2-b534-432e-94fe-7ddc4a617b54 None None] Routers {'24e49d59-ce93-4246-ae1a-cf4080a4b8f5'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:20:02.345932 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-1baa03a2-b534-432e-94fe-7ddc4a617b54 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:19:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['24e49d59-ce93-4246-ae1a-cf4080a4b8f5'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:02.454505 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Deleting port c9840383-de18-463c-8858-5536335f4466 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:20:02.455127 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:02.650899 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-1baa03a2-b534-432e-94fe-7ddc4a617b54 None None] Set router 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 gateway port host: None {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:20:02.810783 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] For port c9840383-de18-463c-8858-5536335f4466, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=c9840383-de18-463c-8858-5536335f4466,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:20:02.811050 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:02.889152 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-1baa03a2-b534-432e-94fe-7ddc4a617b54 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'name': 'tempest-ServerRescueNegativeTestJSON-12783889-router', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:16:04Z', 'updated_at': '2021-05-24T11:20:02Z', 'revision_number': 6, 'project_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'gw_port_host': None}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:02.894848 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1baa03a2-b534-432e-94fe-7ddc4a617b54 None None] The sync data for ids ['24e49d59-ce93-4246-ae1a-cf4080a4b8f5'] in dv-cinder-784092-9-iscsi is: [{'id': '24e49d59-ce93-4246-ae1a-cf4080a4b8f5', 'name': 'tempest-ServerRescueNegativeTestJSON-12783889-router', 'tenant_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:16:04Z', 'updated_at': '2021-05-24T11:20:02Z', 'revision_number': 6, 'project_id': 'bf5fa11a013d4e1a8b86edc5fe49e97b', 'gw_port_host': None}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:20:03.014144 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Calling delete_port for c9840383-de18-463c-8858-5536335f4466 owned by network:router_gateway {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:20:03.223480 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:03.228860 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:03.278859 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Pushing event deleted for resources: {'Port': ['ID=c9840383-de18-463c-8858-5536335f4466,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:03.281617 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:03.566627 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57310) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:03.600590 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-686964'] for router_gateway, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:03.785506 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c0d399-ca3a-4567-897f-94329d013554 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:03.794419 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-684537', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4171'] for router, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:03.798025 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:03.804189 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-19c0d399-ca3a-4567-897f-94329d013554 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:03.808874 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-19c0d399-ca3a-4567-897f-94329d013554 admin admin] Usage tracker for resource:security_group_rule and tenant:0299b0022cb54b1b8373022e8d6ec6c7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:03.812003 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-19c0d399-ca3a-4567-897f-94329d013554 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:03.818846 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-19c0d399-ca3a-4567-897f-94329d013554 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:03.918503 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f0d06b86-d329-4600-9d53-e34aa41f983c req-0c56a141-e8c9-4140-80b8-908b072af33f tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:03.920006 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f0d06b86-d329-4600-9d53-e34aa41f983c req-0c56a141-e8c9-4140-80b8-908b072af33f tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 943 time: 0.4023762 May 24 11:20:03.921174 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-684229'] for router, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:03.921698 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 24e49d59-ce93-4246-ae1a-cf4080a4b8f5 {{(pid=91820) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 11:20:03.957996 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c0d399-ca3a-4567-897f-94329d013554 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:03.962350 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Notify callbacks [] for router, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:03.963969 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c0d399-ca3a-4567-897f-94329d013554 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:03.964285 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] 10.222.0.18 "DELETE /v2.0/routers/24e49d59-ce93-4246-ae1a-cf4080a4b8f5 HTTP/1.1" status: 204 len: 168 time: 2.3916204 May 24 11:20:03.967953 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 57316) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:03.973622 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-710449fd-ea6b-4377-b7ab-19ed9f1f65c4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:04.022372 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-19c0d399-ca3a-4567-897f-94329d013554 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=0299b0022cb54b1b8373022e8d6ec6c7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4545958 May 24 11:20:04.023328 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57322) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:04.024220 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-19c0d399-ca3a-4567-897f-94329d013554 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0ee0031e-a19e-4907-9450-b99e946d9404,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:04.029342 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-19c0d399-ca3a-4567-897f-94329d013554 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:04.092700 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f0d06b86-d329-4600-9d53-e34aa41f983c req-2c7a51e7-c092-4d1a-8326-5f56470dcfa5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1629357 May 24 11:20:04.138237 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b390a5a1-3645-41b5-b7ad-72772d2f6abc admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:04.145079 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_v2 [None req-f250ba27-8f92-496b-b2b4-4db37d4699a2 admin admin] Deleting subnet 52267672-fd8c-45bd-89ea-d9d02a7b249a {{(pid=91820) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:20:04.163268 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f250ba27-8f92-496b-b2b4-4db37d4699a2 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:04.172224 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b390a5a1-3645-41b5-b7ad-72772d2f6abc admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:04.217745 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f250ba27-8f92-496b-b2b4-4db37d4699a2 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534861947'] for subnet, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:04.242726 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b390a5a1-3645-41b5-b7ad-72772d2f6abc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:04.247118 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:04.259430 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b390a5a1-3645-41b5-b7ad-72772d2f6abc admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0ee0031e-a19e-4907-9450-b99e946d9404,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:04.263339 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b390a5a1-3645-41b5-b7ad-72772d2f6abc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:04.267194 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b390a5a1-3645-41b5-b7ad-72772d2f6abc admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:04.280015 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-b390a5a1-3645-41b5-b7ad-72772d2f6abc admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/0ee0031e-a19e-4907-9450-b99e946d9404 HTTP/1.1" status: 204 len: 168 time: 0.2527952 May 24 11:20:04.297330 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f250ba27-8f92-496b-b2b4-4db37d4699a2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:04.475487 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 97fb6682-0049-400a-84fe-731434f40cc6 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:20:04.526590 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 97fb6682-0049-400a-84fe-731434f40cc6 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:20:04.527360 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 97fb6682-0049-400a-84fe-731434f40cc6 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:20:04.528411 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device c9840383-de18-463c-8858-5536335f4466 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:20:04.573208 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: c9840383-de18-463c-8858-5536335f4466 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:20:04.573814 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device c9840383-de18-463c-8858-5536335f4466 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:20:04.782488 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57336) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:04.807203 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:04.832240 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:04.838179 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Usage tracker for resource:security_group_rule and tenant:1e82de011a574610963dc2a51bbd95db is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:04.842090 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:04.852604 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:04.861691 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f250ba27-8f92-496b-b2b4-4db37d4699a2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534788488', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535456750'] for subnet, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:04.870512 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-a35bb4d6-21bc-4e6f-a080-6972ffc74f88 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:04.919846 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f250ba27-8f92-496b-b2b4-4db37d4699a2 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=52267672-fd8c-45bd-89ea-d9d02a7b249a,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:04.923697 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f250ba27-8f92-496b-b2b4-4db37d4699a2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:04.926621 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f250ba27-8f92-496b-b2b4-4db37d4699a2 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:04.942206 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-f250ba27-8f92-496b-b2b4-4db37d4699a2 admin admin] 10.222.0.18 "DELETE /v2.0/subnets/52267672-fd8c-45bd-89ea-d9d02a7b249a HTTP/1.1" status: 204 len: 168 time: 0.9657300 May 24 11:20:04.943270 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 57340) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:04.957793 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:04.964218 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:04.975401 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-0a177670-65d5-403d-8a6b-cd05e0c2c716 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:05.024655 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=1e82de011a574610963dc2a51bbd95db&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2407444 May 24 11:20:05.025718 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57344) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:05.039153 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a1e45600-5e3f-4ae5-a0fe-9f3221e6d84b,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:05.054195 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:05.054897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:05.055445 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:05.056159 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:05.056715 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:05.057355 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:05.058027 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:05.058611 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:05.059197 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:05.059784 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:05.060331 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:05.060929 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:05.072699 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.097s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:05.099090 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.115336 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5c7b2d69-3473-4d8f-b64e-17b0840b42de admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.163382 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5c7b2d69-3473-4d8f-b64e-17b0840b42de admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.228335 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5c7b2d69-3473-4d8f-b64e-17b0840b42de admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.232625 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-dd609af9-08d2-4503-9562-e83785e97e54 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:05.258540 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5c7b2d69-3473-4d8f-b64e-17b0840b42de admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a1e45600-5e3f-4ae5-a0fe-9f3221e6d84b,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:05.264363 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5c7b2d69-3473-4d8f-b64e-17b0840b42de admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:05.277568 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5c7b2d69-3473-4d8f-b64e-17b0840b42de admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.287707 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-5c7b2d69-3473-4d8f-b64e-17b0840b42de admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/a1e45600-5e3f-4ae5-a0fe-9f3221e6d84b HTTP/1.1" status: 204 len: 168 time: 0.2596593 May 24 11:20:05.303339 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.321595 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['4fec6b68-64f1-4553-8ca5-f86c63ac9714']}} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:05.349565 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '6bc6a763-352e-4ddf-bea5-3721ee687b85') {'for_net_delete': True} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:05.349868 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '6bc6a763-352e-4ddf-bea5-3721ee687b85') {} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:05.385048 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534861936'] for segment, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.392498 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 8a7c6451-6355-44ad-a36a-f02ffcfe252c details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:20:05.421494 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.430820 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] Releasing vlan 519 on physical network physnet1 to pool {{(pid=91820) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:20:05.431222 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535262766'] for segment, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: Traceback (most recent call last): May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: timer() May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: cb(*args, **kw) May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: return func(*args, **kwargs) May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: self._notify() May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: self.callback(batched_events) May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: event.method(event) May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: return f(self, *a, **k) May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._get(url).json() May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: return self.request(url, 'GET', **kwargs) May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:20:05.447760 dv-cinder-784092-9-iscsi neutron-server[91820]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:20:05.465125 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fa4c7292-c6e7-48d2-aeb0-442184b3fd78 req-a0081199-a8dc-40bd-a79e-03a838eefa60 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:05.472830 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fa4c7292-c6e7-48d2-aeb0-442184b3fd78 req-a0081199-a8dc-40bd-a79e-03a838eefa60 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 943 time: 0.1350496 May 24 11:20:05.527309 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fa4c7292-c6e7-48d2-aeb0-442184b3fd78 req-cf0bb61b-0cc1-499f-83e3-f75c7f1721d5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0533478 May 24 11:20:05.550929 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.569520 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.575259 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:05.609604 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] Pushing event deleted for resources: {'Network': ['ID=4fec6b68-64f1-4553-8ca5-f86c63ac9714,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:05.613145 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:05.619068 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-c998efb9-406d-470c-8462-0526ebf07e2f admin admin] 10.222.0.18 "DELETE /v2.0/networks/4fec6b68-64f1-4553-8ca5-f86c63ac9714 HTTP/1.1" status: 204 len: 168 time: 0.6708126 May 24 11:20:05.620933 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 8a7c6451-6355-44ad-a36a-f02ffcfe252c, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=8a7c6451-6355-44ad-a36a-f02ffcfe252c,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:20:05.640714 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '8a7c6451-6355-44ad-a36a-f02ffcfe252c', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'port_id': '8a7c6451-6355-44ad-a36a-f02ffcfe252c', 'mac_address': 'fa:16:3e:f3:93:af', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 503, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.5'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:20:05.771681 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.808964 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 8a7c6451-6355-44ad-a36a-f02ffcfe252c {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:20:05.841502 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 8a7c6451-6355-44ad-a36a-f02ffcfe252c, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=8a7c6451-6355-44ad-a36a-f02ffcfe252c,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:20:05.848121 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:05.850709 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:05.934913 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=8a7c6451-6355-44ad-a36a-f02ffcfe252c,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:05.939126 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:05.991987 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57370) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:06.075181 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-49f349fb-395a-4455-b07d-1f8ef5da4101 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=bf5fa11a013d4e1a8b86edc5fe49e97b&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0811379 May 24 11:20:06.078636 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57376) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:06.110878 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['8a7c6451-6355-44ad-a36a-f02ffcfe252c'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:20:06.180672 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e44066e2-4467-4f3b-b358-36703fb96b03 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:06.351606 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e44066e2-4467-4f3b-b358-36703fb96b03 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:06.482838 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e44066e2-4467-4f3b-b358-36703fb96b03 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:06.488919 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:06.511200 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e44066e2-4467-4f3b-b358-36703fb96b03 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=cbdda050-525d-4636-8079-44cfb0e04b03,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:06.519940 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e44066e2-4467-4f3b-b358-36703fb96b03 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:06.573792 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e44066e2-4467-4f3b-b358-36703fb96b03 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:06.575201 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e44066e2-4467-4f3b-b358-36703fb96b03 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/cbdda050-525d-4636-8079-44cfb0e04b03 HTTP/1.1" status: 204 len: 168 time: 0.4948821 May 24 11:20:06.795216 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-264ca5ca-aa5b-444d-a7f1-5d6eac32021b req-37f79d9d-1d79-4fbf-9e1a-8f3b5df7ecf6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:06.798291 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-264ca5ca-aa5b-444d-a7f1-5d6eac32021b req-37f79d9d-1d79-4fbf-9e1a-8f3b5df7ecf6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 944 time: 0.1252403 May 24 11:20:06.878810 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-264ca5ca-aa5b-444d-a7f1-5d6eac32021b req-de9bef1b-5087-42af-9671-3c91aac36feb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0687962 May 24 11:20:07.421773 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 8a7c6451-6355-44ad-a36a-f02ffcfe252c up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:20:07.550192 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 8a7c6451-6355-44ad-a36a-f02ffcfe252c completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:20:07.561430 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 8a7c6451-6355-44ad-a36a-f02ffcfe252c triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:20:07.562160 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:08.056113 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:08.221288 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '8a7c6451-6355-44ad-a36a-f02ffcfe252c'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:20:08.249860 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 8a7c6451-6355-44ad-a36a-f02ffcfe252c, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=8a7c6451-6355-44ad-a36a-f02ffcfe252c,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:20:08.255308 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:08.257435 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:08.339555 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=8a7c6451-6355-44ad-a36a-f02ffcfe252c,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:08.343677 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:08.383196 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '8a7c6451-6355-44ad-a36a-f02ffcfe252c', 'code': 200} May 24 11:20:08.456375 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d61d9e9-4ef2-4e73-8cbf-f9c36a45647f req-c4a8ec14-af6c-432c-b0d6-5c440d1d254d tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:08.464544 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6d61d9e9-4ef2-4e73-8cbf-f9c36a45647f req-c4a8ec14-af6c-432c-b0d6-5c440d1d254d tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 945 time: 0.2033150 May 24 11:20:08.560263 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6d61d9e9-4ef2-4e73-8cbf-f9c36a45647f req-ab359aca-30dc-4ea7-a892-8f4b75654a95 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0929594 May 24 11:20:08.604366 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 57460) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:08.835109 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeShelveTestJSON-984022173-network', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:08.844461 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:08.850216 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Usage tracker for resource:network and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:08.857154 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:08.868539 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:08.906673 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Made reservation on behalf of 05ec02ff35d2402e8d7bf064752e67ea for: {'network': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:20:08.910457 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:08.928372 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:08.960359 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:08.969388 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Usage tracker for resource:security_group_rule and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:08.979634 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:08.994054 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:09.093816 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-327fd1dd-5663-47b6-92cb-9357dcc354e2 req-21f55b4b-4cb3-4d54-a172-ec45e885eee6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:09.095333 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-327fd1dd-5663-47b6-92cb-9357dcc354e2 req-21f55b4b-4cb3-4d54-a172-ec45e885eee6 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 1098 time: 0.3406587 May 24 11:20:09.179650 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-327fd1dd-5663-47b6-92cb-9357dcc354e2 req-1813718b-c1f1-43df-8d49-b8bd93e8ba1f service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0704317 May 24 11:20:09.249012 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6b4bb82-8804-4fb1-b1fb-2dc7d58c6b2c req-8cd51216-41fb-4e65-98b8-9d2c8324da32 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:09.250690 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6b4bb82-8804-4fb1-b1fb-2dc7d58c6b2c req-8cd51216-41fb-4e65-98b8-9d2c8324da32 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 945 time: 0.1904404 May 24 11:20:09.296288 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:09.308620 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:09.321777 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f15036e6-45b0-41b8-a8a4-4ad211f15e23 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:09.342146 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] vlan segment allocate from pool success with {'vlan_id': 500, 'physical_network': 'physnet1'} {{(pid=91820) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:20:09.352548 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6b4bb82-8804-4fb1-b1fb-2dc7d58c6b2c req-197f2b0c-33f3-4bd2-8b42-74b52cdf64e9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0942700 May 24 11:20:09.379668 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2361ea90-9180-4574-b91e-b236f5fc2de0,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:09.383188 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57486) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:09.387397 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:09.387765 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.segments_db [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Added segment 31ae7943-96d7-4018-b76d-057ae76567e2 of type vlan for network 7877e643-fe9e-4041-91fa-3a4f44279fd2 May 24 11:20:09.389879 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:09.390265 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:09.390645 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:09.391193 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:09.391744 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:09.392276 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:09.392908 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:09.393216 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:09.393555 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:09.394321 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:09.394891 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:09.395436 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:09.419387 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:09.465521 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:09.539909 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:09.542750 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:09.544713 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:09.555188 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e6b99963-932f-45cd-bfa7-db5371492dec admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:09.642030 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-682ed575-4a46-4319-a0e7-f56d94c8d552 req-86d52474-05ef-428d-b671-d54e79e56735 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:09.651977 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-682ed575-4a46-4319-a0e7-f56d94c8d552 req-86d52474-05ef-428d-b671-d54e79e56735 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1450939 May 24 11:20:09.721331 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 889 time: 1.1151972 May 24 11:20:09.725879 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 57506) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:09.743432 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Request body: {'subnet': {'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-subnet', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'enable_dhcp': True, 'ip_version': 4}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:09.755523 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:09.757766 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-682ed575-4a46-4319-a0e7-f56d94c8d552 req-56f7ec90-ba26-4f29-bfef-1f51a327d725 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1045170 May 24 11:20:09.765322 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Usage tracker for resource:subnet and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:09.798676 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Pushing event updated for resources: {'Network': ['ID=7877e643-fe9e-4041-91fa-3a4f44279fd2,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:09.809389 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network 7877e643-fe9e-4041-91fa-3a4f44279fd2 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:20:09.810404 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:09.816821 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ab255c1c-0ca4-49a6-81a9-6282b9f272c4 tempest-TestMinimumBasicScenario-147190028 tempest-TestMinimumBasicScenario-147190028-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:09.817130 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-ab255c1c-0ca4-49a6-81a9-6282b9f272c4 tempest-TestMinimumBasicScenario-147190028 tempest-TestMinimumBasicScenario-147190028-project-admin] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 1093 time: 0.4326000 May 24 11:20:09.818401 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:09.821834 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57516) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:09.827311 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.273s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:09.830953 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Request body: {'floatingip': {'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'port_id': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'fixed_ip_address': '172.20.0.6'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:09.834671 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Made reservation on behalf of 05ec02ff35d2402e8d7bf064752e67ea for: {'subnet': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:20:09.837680 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-535456753'] for subnet, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:09.841020 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:09.847600 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Usage tracker for resource:floatingip and tenant:788050c389224d17b163f24bb67902bf is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:09.852157 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:09.861762 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:09.875436 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for floatingip, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:10.042240 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f62e6c2-e56e-489c-98dc-8ef534bfe2fc req-e8af3e39-3cfb-4c6e-bd17-e84778e4d3cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:10.043792 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1f62e6c2-e56e-489c-98dc-8ef534bfe2fc req-e8af3e39-3cfb-4c6e-bd17-e84778e4d3cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 945 time: 0.2854621 May 24 11:20:10.099177 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:10.130435 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1f62e6c2-e56e-489c-98dc-8ef534bfe2fc req-b1e3a75a-3f6e-4b0b-bea0-a9997e8f290d tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0759306 May 24 11:20:10.258001 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Allocated IP 172.27.0.236 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/68eb8843-6cde-414a-9854-2653eea784b1) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:20:10.396162 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-534788555', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-535456743'] for subnet, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:10.401253 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f250ba27-8f92-496b-b2b4-4db37d4699a2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:10.419318 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:10.455424 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:10.460265 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c64b765b-c30d-477f-ad9e-c44de7904ffe service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:10.474716 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Pushing event updated for resources: {'Subnet': ['ID=12ff35e8-9af8-4339-95c9-c62c90d8c6fd,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:10.478455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:10.483183 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Checking segment: {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:20:10.483533 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Network hostable DHCP agents. Network: 7877e643-fe9e-4041-91fa-3a4f44279fd2, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91820) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:20:10.501884 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Network 7877e643-fe9e-4041-91fa-3a4f44279fd2 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91820) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:20:10.516725 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, 7877e643-fe9e-4041-91fa-3a4f44279fd2) {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:20:10.522933 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:20:10.526137 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:20:10.526473 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:10.527046 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:10.529963 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-8d6859e1-d17f-4372-8688-447821fcf5ab None None] Network 7877e643-fe9e-4041-91fa-3a4f44279fd2 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:20:10.530110 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:20:10.560541 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-3ee01b9c-420e-4f24-9a54-f5444412f92f admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 852 time: 0.8331823 May 24 11:20:10.563138 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 57520) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:10.568478 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Request body: {'router': {'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:10.570336 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event updated for resources: {'Port': ['ID=68eb8843-6cde-414a-9854-2653eea784b1,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:10.572963 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:10.574849 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:10.577163 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Usage tracker for resource:router and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:10.580332 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:10.586847 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:10.599585 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-684314', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4157'] for router, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:10.600675 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-684570', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-687045', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-715560', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4164'] for router, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:10.618511 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:10.776621 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpea482e27-f15a-5756-a111-7e0dba035002-7877e643-fe9e-4041-91fa-3a4f44279fd2', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd'}]}} from dv-cinder-784092-9-iscsi. {{(pid=91821) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 11:20:10.777848 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:10.922030 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-687072', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-684328', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4178'] for router, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:11.109483 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.db_base_plugin_common [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Allocated IP 172.20.0.2 (7877e643-fe9e-4041-91fa-3a4f44279fd2/12ff35e8-9af8-4339-95c9-c62c90d8c6fd/9d9e4487-8f1c-4ba1-88e1-16045be7df72) {{(pid=91821) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:20:11.155385 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:11.195172 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-958ba466-ed6e-4766-9219-df717a3e36e3 req-331adf0b-709d-4bcb-aa0b-096eb855f5e5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:11.203576 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-958ba466-ed6e-4766-9219-df717a3e36e3 req-331adf0b-709d-4bcb-aa0b-096eb855f5e5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1467633 May 24 11:20:11.352007 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-958ba466-ed6e-4766-9219-df717a3e36e3 req-d196c895-82f5-487c-b635-1fd742fb3813 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1473169 May 24 11:20:11.380233 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:11.524359 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:11.560804 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:11.566860 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Notify callbacks [] for port, precommit_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:11.582608 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Transition to ACTIVE for port object 9d9e4487-8f1c-4ba1-88e1-16045be7df72 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:20:11.611104 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Transition to ACTIVE for port object 9d9e4487-8f1c-4ba1-88e1-16045be7df72 will not be triggered until provisioned by entity DHCP. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:20:11.624119 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:11.629736 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b53b0216-0c0d-4cc4-932c-258ec1c9f4c2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:11.636149 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf100a2d-63b6-4723-ad88-ae73fb6f01bf req-4a3abe69-96d4-404b-9741-5f8525830ccc tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:11.638054 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-bf100a2d-63b6-4723-ad88-ae73fb6f01bf req-4a3abe69-96d4-404b-9741-5f8525830ccc tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 945 time: 0.2471342 May 24 11:20:11.698321 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Network 7877e643-fe9e-4041-91fa-3a4f44279fd2 is already hosted by enough agents. {{(pid=91821) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:20:11.701598 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.base_scheduler [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Resources bound (agent type, host, resource id): {{(pid=91821) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:20:11.705497 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Received port after_create {{(pid=91821) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:20:11.712310 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Pushing event updated for resources: {'Port': ['ID=9d9e4487-8f1c-4ba1-88e1-16045be7df72,revision_number=1']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:11.716571 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:11.735835 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-bf100a2d-63b6-4723-ad88-ae73fb6f01bf req-2ed3a0bc-ba77-4e4b-b7a3-2382f4be8b40 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0905106 May 24 11:20:11.810570 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:11.817708 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:11.822849 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Allocated IP 172.27.0.76 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/839b9282-2a46-4561-9d08-f49bad628dc7) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:20:11.825221 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Attempting to bind port 9d9e4487-8f1c-4ba1-88e1-16045be7df72 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:20:11.826032 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Attempting to bind port 9d9e4487-8f1c-4ba1-88e1-16045be7df72 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:20:11.826368 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:11.826550 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Attempting to bind port 9d9e4487-8f1c-4ba1-88e1-16045be7df72 on network 7877e643-fe9e-4041-91fa-3a4f44279fd2 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:20:11.834027 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 19, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:20:11.834506 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Checking segment: {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:20:11.835078 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:11.835597 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Bound using segment: {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:20:11.836258 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Bound port: 9d9e4487-8f1c-4ba1-88e1-16045be7df72, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:20:11.836793 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:12.017471 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] For port 9d9e4487-8f1c-4ba1-88e1-16045be7df72, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:20:12.022121 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] For port 9d9e4487-8f1c-4ba1-88e1-16045be7df72, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=9d9e4487-8f1c-4ba1-88e1-16045be7df72,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:20:12.027720 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:12.033027 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:12.073660 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:12.115167 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:12.132853 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d6ca27b8-48c1-4f4c-81c9-e1c23b9a47b7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:12.141046 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Pushing event updated for resources: {'Port': ['ID=9d9e4487-8f1c-4ba1-88e1-16045be7df72,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:12.145454 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:12.158307 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event updated for resources: {'Port': ['ID=68eb8843-6cde-414a-9854-2653eea784b1,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:12.177865 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.349s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:12.177865 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:12.240111 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:20:12.246386 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:20:12.252131 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:20:12.320727 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Pushing event updated for resources: {'Port': ['ID=839b9282-2a46-4561-9d08-f49bad628dc7,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:12.327069 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.194s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:12.392330 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1cd88cc9-f910-4baa-abe3-036936fb7d27 req-55162d95-14b6-48f1-bb02-e62fb2d082b4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:12.404134 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1cd88cc9-f910-4baa-abe3-036936fb7d27 req-55162d95-14b6-48f1-bb02-e62fb2d082b4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 945 time: 0.3564613 May 24 11:20:12.520186 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Floating IP 02886571-aeff-4f23-8fb1-dd7a8651636b associated. External IP: 172.27.0.236, port: e0dbce52-01cb-4042-b340-ea0a6b7da010. May 24 11:20:12.580806 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1cd88cc9-f910-4baa-abe3-036936fb7d27 req-8818fa58-08b1-4784-8686-44e06f88e2c1 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1757169 May 24 11:20:12.640369 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57596) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:12.831669 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b10f9a3-9e8f-437e-98fb-3537f898d27a req-78f031d2-dd51-4546-afa9-66d605a6d52a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:12.833259 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9b10f9a3-9e8f-437e-98fb-3537f898d27a req-78f031d2-dd51-4546-afa9-66d605a6d52a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.2775974 May 24 11:20:12.957226 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Router 684315b8-ba5e-4d23-8963-64aca3b067cb has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:20:12.966871 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-7b6681e9-0ec2-4998-a3f9-33227ac85e52 tempest-TestVolumeBootPattern-1615708037 tempest-TestVolumeBootPattern-1615708037-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:12.972278 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-7b6681e9-0ec2-4998-a3f9-33227ac85e52 tempest-TestVolumeBootPattern-1615708037 tempest-TestVolumeBootPattern-1615708037-project-admin] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 1093 time: 0.3276331 May 24 11:20:12.973429 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 57602) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:12.983472 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'floatingip': {'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'port_id': '8a7c6451-6355-44ad-a36a-f02ffcfe252c', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address': '172.20.0.5'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:12.990095 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9b10f9a3-9e8f-437e-98fb-3537f898d27a req-cc0c99e9-95fb-433b-9e12-994232c5b455 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1010675 May 24 11:20:13.010006 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:13.018871 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Usage tracker for resource:floatingip and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:13.023618 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:13.032730 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:13.038504 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.166040 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.185679 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:20:13.202673 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-540ce900-8cfd-4c19-901d-c695995cb5b1 None None] Sync routers for ids ['684315b8-ba5e-4d23-8963-64aca3b067cb'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:20:13.216039 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.219852 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-540ce900-8cfd-4c19-901d-c695995cb5b1 None None] Routers {'684315b8-ba5e-4d23-8963-64aca3b067cb'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:20:13.220481 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-540ce900-8cfd-4c19-901d-c695995cb5b1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:19:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['684315b8-ba5e-4d23-8963-64aca3b067cb'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:13.288930 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 996 time: 3.4621229 May 24 11:20:13.288930 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:20:13.291003 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57606) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:13.297432 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-a5dbc980-34d2-4342-83a2-1e903cfbca19 None None] Network 7877e643-fe9e-4041-91fa-3a4f44279fd2 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:20:13.310771 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-1726653752', 'description': 'tempest-secgroup-smoke-1726653752 description', 'project_id': '788050c389224d17b163f24bb67902bf'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:13.318095 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:13.330713 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:13.343704 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.354817 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'status': 'completed', 'code': 200} May 24 11:20:13.369026 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:13.373343 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Usage tracker for resource:security_group_rule and tenant:788050c389224d17b163f24bb67902bf is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:13.377929 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:13.385640 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:13.474964 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.487060 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-2d3b25c9-9206-4db8-aa3c-36257e72a479 None None] Provisioning for port 9d9e4487-8f1c-4ba1-88e1-16045be7df72 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:20:13.488378 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.493742 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5c7b2d69-3473-4d8f-b64e-17b0840b42de admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:13.550479 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.573749 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event updated for resources: {'SecurityGroup': ['ID=9286a221-16dd-4047-bf8c-6373837405d0,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:13.577142 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:13.587518 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.593510 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1794 time: 0.2830760 May 24 11:20:13.593510 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57616) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:13.595451 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-684307', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-687011'] for router, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.601743 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Request body: {'security_group_rule': {'security_group_id': '9286a221-16dd-4047-bf8c-6373837405d0', 'project_id': '788050c389224d17b163f24bb67902bf', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:13.609908 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:13.615354 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Usage tracker for resource:security_group_rule and tenant:788050c389224d17b163f24bb67902bf is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:13.618996 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:13.625741 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:13.656432 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.668010 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-540ce900-8cfd-4c19-901d-c695995cb5b1 None None] Set router 684315b8-ba5e-4d23-8963-64aca3b067cb gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:20:13.721015 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.726245 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.728169 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:13.751128 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=6e1d5ba6-7e31-4d12-bc1c-d38df3b467a5,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:13.755578 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:13.763446 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.764704 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.1699293 May 24 11:20:13.767494 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57620) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:13.773470 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Request body: {'security_group_rule': {'security_group_id': '9286a221-16dd-4047-bf8c-6373837405d0', 'project_id': '788050c389224d17b163f24bb67902bf', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:13.779350 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:13.783227 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Usage tracker for resource:security_group_rule and tenant:788050c389224d17b163f24bb67902bf is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:13.786179 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:13.792053 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:13.807834 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.871289 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Allocated IP 172.27.0.211 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/0cf9a14b-892a-4235-8708-1f72096a9a29) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:20:13.872623 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.879469 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.882439 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e1d81d45-4d5b-400d-ab4b-478d7372d6a2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:13.902981 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=a8161295-be16-40dc-9b60-288d196e05dd,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:13.907302 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:13.917960 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:13.919408 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.1508048 May 24 11:20:13.923280 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57624) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:13.929197 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-ba5754fe-be82-4e19-a3fa-dc9094da6ed3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Request body: {'security_group_rule': {'security_group_id': '9286a221-16dd-4047-bf8c-6373837405d0', 'project_id': '788050c389224d17b163f24bb67902bf', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:13.935197 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-ba5754fe-be82-4e19-a3fa-dc9094da6ed3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:13.939013 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-ba5754fe-be82-4e19-a3fa-dc9094da6ed3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Usage tracker for resource:security_group_rule and tenant:788050c389224d17b163f24bb67902bf is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:13.941942 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-ba5754fe-be82-4e19-a3fa-dc9094da6ed3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:13.952661 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-ba5754fe-be82-4e19-a3fa-dc9094da6ed3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:13.982374 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ba5754fe-be82-4e19-a3fa-dc9094da6ed3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.145729 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-540ce900-8cfd-4c19-901d-c695995cb5b1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:18:39Z', 'revision_number': 4, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '02886571-aeff-4f23-8fb1-dd7a8651636b', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'floating_ip_address': '172.27.0.236', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'port_id': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'fixed_ip_address': '172.20.0.6', 'status': 'DOWN', 'standard_attr_id': 437, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'mac_address': 'fa:16:3e:b3:e8:d3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'fa7a9207-daa9-4469-9eec-51039940736a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:10Z', 'revision_number': 0, 'project_id': '788050c389224d17b163f24bb67902bf', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '90af9233-b13f-40e5-a0f8-872fe40ddfed', 'name': '', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'mac_address': 'fa:16:3e:ab:4f:f6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_interface', 'standard_attr_id': 356, 'fixed_ips': [{'subnet_id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:38Z', 'updated_at': '2021-05-24T11:18:52Z', 'revision_number': 5, 'project_id': '788050c389224d17b163f24bb67902bf', 'subnets': [{'id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:14.151349 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-540ce900-8cfd-4c19-901d-c695995cb5b1 None None] Checking router: 684315b8-ba5e-4d23-8963-64aca3b067cb for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:20:14.163918 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-540ce900-8cfd-4c19-901d-c695995cb5b1 None None] Routers {'684315b8-ba5e-4d23-8963-64aca3b067cb'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:20:14.164711 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-540ce900-8cfd-4c19-901d-c695995cb5b1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:19:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['684315b8-ba5e-4d23-8963-64aca3b067cb'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:14.236038 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Router c2ffed60-7ad5-4215-9e01-680707fb98d6 is scheduled to L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 with binding_index 1 {{(pid=91820) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 11:20:14.267744 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ba5754fe-be82-4e19-a3fa-dc9094da6ed3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.278533 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ba5754fe-be82-4e19-a3fa-dc9094da6ed3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.280681 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:14.304004 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ba5754fe-be82-4e19-a3fa-dc9094da6ed3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=ace49203-346a-4f3b-b30a-7634a7f9099a,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:14.308894 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ba5754fe-be82-4e19-a3fa-dc9094da6ed3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:14.314687 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ba5754fe-be82-4e19-a3fa-dc9094da6ed3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.316511 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-ba5754fe-be82-4e19-a3fa-dc9094da6ed3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.3922548 May 24 11:20:14.319348 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57630) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:14.325963 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-b4fc5b4a-590f-45eb-93b5-c7a2d3e21db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Request body: {'security_group_rule': {'security_group_id': '9286a221-16dd-4047-bf8c-6373837405d0', 'project_id': '788050c389224d17b163f24bb67902bf', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:14.332163 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.332868 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-b4fc5b4a-590f-45eb-93b5-c7a2d3e21db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:14.337589 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-b4fc5b4a-590f-45eb-93b5-c7a2d3e21db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Usage tracker for resource:security_group_rule and tenant:788050c389224d17b163f24bb67902bf is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:14.340759 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-b4fc5b4a-590f-45eb-93b5-c7a2d3e21db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:14.347464 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-b4fc5b4a-590f-45eb-93b5-c7a2d3e21db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:14.372979 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b4fc5b4a-590f-45eb-93b5-c7a2d3e21db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.409441 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-540ce900-8cfd-4c19-901d-c695995cb5b1 None None] Set router 684315b8-ba5e-4d23-8963-64aca3b067cb gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:20:14.466591 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c9af42b9-7572-413e-b005-b69b5341e96b req-ceaa12fe-e37a-470a-a01b-586d650961e0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:14.468149 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b4fc5b4a-590f-45eb-93b5-c7a2d3e21db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.469269 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c9af42b9-7572-413e-b005-b69b5341e96b req-ceaa12fe-e37a-470a-a01b-586d650961e0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.3097813 May 24 11:20:14.478158 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b4fc5b4a-590f-45eb-93b5-c7a2d3e21db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.486786 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-01bae3ed-b1dd-40a2-8013-38f28d35bb3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:14.523525 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b4fc5b4a-590f-45eb-93b5-c7a2d3e21db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=d0782bf3-9b18-439c-8c24-e8668d162639,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:14.526801 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b4fc5b4a-590f-45eb-93b5-c7a2d3e21db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:14.537710 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b4fc5b4a-590f-45eb-93b5-c7a2d3e21db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.542642 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-b4fc5b4a-590f-45eb-93b5-c7a2d3e21db4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.2190123 May 24 11:20:14.546881 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57634) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:14.553078 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c9af42b9-7572-413e-b005-b69b5341e96b req-9eefd8a9-f674-404a-95fc-d03af17021a1 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0687263 May 24 11:20:14.554492 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:20:14.555188 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Request body: {'security_group_rule': {'security_group_id': '9286a221-16dd-4047-bf8c-6373837405d0', 'project_id': '788050c389224d17b163f24bb67902bf', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:14.566457 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Sync routers for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:20:14.570509 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 9d9e4487-8f1c-4ba1-88e1-16045be7df72 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:20:14.571938 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:14.582351 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:20:14.584657 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:19:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:14.611644 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Usage tracker for resource:security_group_rule and tenant:788050c389224d17b163f24bb67902bf is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:14.618590 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:14.622368 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.626262 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:14.641697 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.securitygroups_db [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Project 788050c389224d17b163f24bb67902bf added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 24 11:20:14.662534 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.696591 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0759abe1-2c28-411e-985a-24b91bb60eea tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:14.738894 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.744617 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.761347 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b499e0db-80a2-4592-968f-4c23568736b9 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:14.781449 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=5c49768a-95f4-45c6-8dfc-db57123ab431,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:14.785637 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:14.791176 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.795614 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.2463946 May 24 11:20:14.797128 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57642) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:14.804776 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Request body: {'security_group_rule': {'security_group_id': '9286a221-16dd-4047-bf8c-6373837405d0', 'project_id': '788050c389224d17b163f24bb67902bf', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:14.810285 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:14.814398 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Usage tracker for resource:security_group_rule and tenant:788050c389224d17b163f24bb67902bf is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:14.814917 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] Notify callbacks [] for router, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.816198 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-82888aec-7cb1-47a5-8112-890025abb530 admin admin] 10.222.0.18 "POST /v2.0/routers HTTP/1.1" status: 201 len: 927 time: 4.2521205 May 24 11:20:14.821338 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:14.827824 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:14.829157 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 57646) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:14.839964 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 9d9e4487-8f1c-4ba1-88e1-16045be7df72 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:20:14.843874 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.securitygroups_db [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Project 788050c389224d17b163f24bb67902bf added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 24 11:20:14.849185 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 9d9e4487-8f1c-4ba1-88e1-16045be7df72 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:20:14.849572 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.852059 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.874859 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:20:14.952838 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.958284 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:14.968703 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:14.981700 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:20:14.992472 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=af4cc946-3f02-4b64-b0b9-8c7a77987c3d,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:14.994914 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:20:14.996264 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:15.008097 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:20:15.010689 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:15.015606 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.2183156 May 24 11:20:15.040100 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'Port': ['ID=0cf9a14b-892a-4235-8708-1f72096a9a29,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:15.053752 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:15.054792 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.358s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:15.095082 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2fd7087-17cb-409f-9d31-c293469676bd req-cd081fa3-3bb1-4f3e-9030-8ee086df4646 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:15.097246 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d2fd7087-17cb-409f-9d31-c293469676bd req-cd081fa3-3bb1-4f3e-9030-8ee086df4646 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=788050c389224d17b163f24bb67902bf&device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 1098 time: 0.4763639 May 24 11:20:15.106922 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-540ce900-8cfd-4c19-901d-c695995cb5b1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:18:39Z', 'revision_number': 4, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '02886571-aeff-4f23-8fb1-dd7a8651636b', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'floating_ip_address': '172.27.0.236', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'port_id': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'fixed_ip_address': '172.20.0.6', 'status': 'DOWN', 'standard_attr_id': 437, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'mac_address': 'fa:16:3e:b3:e8:d3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'fa7a9207-daa9-4469-9eec-51039940736a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:10Z', 'revision_number': 0, 'project_id': '788050c389224d17b163f24bb67902bf', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '90af9233-b13f-40e5-a0f8-872fe40ddfed', 'name': '', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'mac_address': 'fa:16:3e:ab:4f:f6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_interface', 'standard_attr_id': 356, 'fixed_ips': [{'subnet_id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:38Z', 'updated_at': '2021-05-24T11:18:52Z', 'revision_number': 5, 'project_id': '788050c389224d17b163f24bb67902bf', 'subnets': [{'id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:15.113811 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 9d9e4487-8f1c-4ba1-88e1-16045be7df72, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=9d9e4487-8f1c-4ba1-88e1-16045be7df72,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:20:15.115817 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-540ce900-8cfd-4c19-901d-c695995cb5b1 None None] The sync data for ids ['684315b8-ba5e-4d23-8963-64aca3b067cb'] in dv-cinder-784092-9-iscsi is: [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:18:39Z', 'revision_number': 4, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '02886571-aeff-4f23-8fb1-dd7a8651636b', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'floating_ip_address': '172.27.0.236', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'port_id': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'fixed_ip_address': '172.20.0.6', 'status': 'DOWN', 'standard_attr_id': 437, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'mac_address': 'fa:16:3e:b3:e8:d3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'fa7a9207-daa9-4469-9eec-51039940736a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:10Z', 'revision_number': 0, 'project_id': '788050c389224d17b163f24bb67902bf', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '90af9233-b13f-40e5-a0f8-872fe40ddfed', 'name': '', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'mac_address': 'fa:16:3e:ab:4f:f6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_interface', 'standard_attr_id': 356, 'fixed_ips': [{'subnet_id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:38Z', 'updated_at': '2021-05-24T11:18:52Z', 'revision_number': 5, 'project_id': '788050c389224d17b163f24bb67902bf', 'subnets': [{'id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:20:15.121393 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:15.123529 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:15.171532 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-3dab32e6-f4c7-4553-8de2-53b73705d223 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Synchronizing usage tracker for tenant:788050c389224d17b163f24bb67902bf on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:20:15.195239 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-3dab32e6-f4c7-4553-8de2-53b73705d223 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Unset dirty status for tenant:788050c389224d17b163f24bb67902bf on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:20:15.196766 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-3dab32e6-f4c7-4553-8de2-53b73705d223 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-1726653752&fields=id&tenant_id=788050c389224d17b163f24bb67902bf HTTP/1.1" status: 200 len: 267 time: 0.0922911 May 24 11:20:15.207134 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=9d9e4487-8f1c-4ba1-88e1-16045be7df72,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:15.210386 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:15.307377 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:12Z', 'revision_number': 3, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:11Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:15.311408 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Checking router: c2ffed60-7ad5-4215-9e01-680707fb98d6 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:20:15.311765 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Updating router c2ffed60-7ad5-4215-9e01-680707fb98d6 port 839b9282-2a46-4561-9d08-f49bad628dc7 binding host dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:20:15.316624 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-3f562c4c-f46e-4a6f-8965-9da6b5e4203e tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:15.318264 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-3f562c4c-f46e-4a6f-8965-9da6b5e4203e tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 945 time: 0.1146739 May 24 11:20:15.330000 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-756b0c24-ce57-456e-a712-a0b02326bc4b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Request body: {'port': {'security_groups': ['29c10cc8-55e7-4791-971b-485a28ba237e', '9286a221-16dd-4047-bf8c-6373837405d0']}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:15.368493 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:15.397757 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8b58b596-0291-4271-b30f-181e4353c3d0 None None] New status for floating IP 02886571-aeff-4f23-8fb1-dd7a8651636b: ACTIVE {{(pid=91821) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:20:15.480538 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-756b0c24-ce57-456e-a712-a0b02326bc4b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:15.793164 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-756b0c24-ce57-456e-a712-a0b02326bc4b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] For port e0dbce52-01cb-4042-b340-ea0a6b7da010, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:20:15.819974 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-756b0c24-ce57-456e-a712-a0b02326bc4b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:15.826026 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-756b0c24-ce57-456e-a712-a0b02326bc4b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:15.837734 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:15.853662 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['e0dbce52-01cb-4042-b340-ea0a6b7da010'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:20:15.858003 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Notify callbacks [] for port, precommit_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:15.892670 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.provisioning_blocks [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Transition to ACTIVE for port object 839b9282-2a46-4561-9d08-f49bad628dc7 will not be triggered until provisioned by entity L2. {{(pid=91822) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:20:15.903642 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:15.911162 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:15.983832 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 57660) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:15.993214 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-756b0c24-ce57-456e-a712-a0b02326bc4b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event updated for resources: {'Port': ['ID=e0dbce52-01cb-4042-b340-ea0a6b7da010,revision_number=6']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:15.996609 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-756b0c24-ce57-456e-a712-a0b02326bc4b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.159s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:16.011439 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-756b0c24-ce57-456e-a712-a0b02326bc4b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:16.011874 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-756b0c24-ce57-456e-a712-a0b02326bc4b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:16.013438 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-63f55c0f-afd8-4066-b553-fd17fd462e2e req-756b0c24-ce57-456e-a712-a0b02326bc4b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "PUT /v2.0/ports/e0dbce52-01cb-4042-b340-ea0a6b7da010 HTTP/1.1" status: 200 len: 981 time: 0.6888521 May 24 11:20:16.014061 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:20:16.036947 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2fd7087-17cb-409f-9d31-c293469676bd req-e81a8747-0985-403c-8dd7-ec22e406e5fe service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:16.038745 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d2fd7087-17cb-409f-9d31-c293469676bd req-e81a8747-0985-403c-8dd7-ec22e406e5fe service neutron] 10.222.0.93 "GET /v2.0/networks?id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 HTTP/1.1" status: 200 len: 915 time: 0.9300189 May 24 11:20:16.055861 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Attempting to bind port 839b9282-2a46-4561-9d08-f49bad628dc7 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:20:16.056384 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Attempting to bind port 839b9282-2a46-4561-9d08-f49bad628dc7 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:20:16.056850 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Attempting to bind port 839b9282-2a46-4561-9d08-f49bad628dc7 on network 858b8fad-a73f-4578-9079-6e731b9ff7e2 {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:20:16.066575 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Pushing event updated for resources: {'Port': ['ID=839b9282-2a46-4561-9d08-f49bad628dc7,revision_number=2']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:16.069261 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 19, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:20:16.069834 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Checking segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91822) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:20:16.070386 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:16.070822 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Bound using segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:20:16.072977 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Bound port: 839b9282-2a46-4561-9d08-f49bad628dc7, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:20:16.073532 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:16.113429 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['e0dbce52-01cb-4042-b340-ea0a6b7da010'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:20:16.126729 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:16.127220 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:16.127653 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:16.128614 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:16.129036 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:16.129508 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:16.130483 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:16.130959 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:16.131515 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:16.131941 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:16.132363 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:16.132780 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:20:16.159935 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.248s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:16.203300 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] For port 839b9282-2a46-4561-9d08-f49bad628dc7, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91822) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:20:16.212559 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] For port 839b9282-2a46-4561-9d08-f49bad628dc7, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=839b9282-2a46-4561-9d08-f49bad628dc7,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91822) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:20:16.220807 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:16.223768 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:16.303762 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d2fd7087-17cb-409f-9d31-c293469676bd req-b5ad9221-6fac-47a2-82d0-844545c9e627 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010 HTTP/1.1" status: 200 len: 1001 time: 0.1585913 May 24 11:20:16.333760 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:20:16.334749 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:19:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:16.347984 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-549777d2-9987-43cd-80a3-8982668e84dc req-6f184864-6cd9-4698-a32b-8b0f74a11219 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:16.350293 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-549777d2-9987-43cd-80a3-8982668e84dc req-6f184864-6cd9-4698-a32b-8b0f74a11219 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.1805766 May 24 11:20:16.417598 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-549777d2-9987-43cd-80a3-8982668e84dc req-3c54b131-790e-4848-aa36-7bccd6bf7f02 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0605092 May 24 11:20:16.492759 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-472d9a8d-a0c8-4f59-9fd0-ddafe164019a None None] Provisioning complete for port e0dbce52-01cb-4042-b340-ea0a6b7da010 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:20:16.493157 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-472d9a8d-a0c8-4f59-9fd0-ddafe164019a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:16.531782 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'status': 'completed', 'code': 200} May 24 11:20:16.558298 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:20:16.566248 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_v2 [None req-b44e6310-1302-4363-ae7d-5eca7b23c4e3 admin admin] Deleting subnet 1ed78d41-8564-48a2-acc1-2ef21df88d9e {{(pid=91820) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:20:16.584006 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b44e6310-1302-4363-ae7d-5eca7b23c4e3 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:16.589741 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 9d9e4487-8f1c-4ba1-88e1-16045be7df72 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:20:16.763915 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Pushing event updated for resources: {'Port': ['ID=839b9282-2a46-4561-9d08-f49bad628dc7,revision_number=3']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:16.768494 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.545s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:16.884259 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b44e6310-1302-4363-ae7d-5eca7b23c4e3 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534861947'] for subnet, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:16.884536 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 9d9e4487-8f1c-4ba1-88e1-16045be7df72 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:20:16.885043 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:16.893323 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:12Z', 'revision_number': 3, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:16Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:16.897839 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c2491de9-f1e3-4a31-b8eb-686493822bde None None] The sync data for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi is: [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:12Z', 'revision_number': 3, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:16Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:20:16.941958 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2fd7087-17cb-409f-9d31-c293469676bd req-74b5ee85-364e-4ea7-ac56-e4e724c23e43 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:16.943661 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d2fd7087-17cb-409f-9d31-c293469676bd req-74b5ee85-364e-4ea7-ac56-e4e724c23e43 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=6a80c579-20d5-46c7-8c07-6438f1367afd HTTP/1.1" status: 200 len: 853 time: 0.6246021 May 24 11:20:17.037261 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5bd8ac89-d50e-40ba-b7a1-761f6ea08cee req-8d2ebf33-ba57-49e3-9389-dc87b9812ad6 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:17.038769 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5bd8ac89-d50e-40ba-b7a1-761f6ea08cee req-8d2ebf33-ba57-49e3-9389-dc87b9812ad6 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 1.0054014 May 24 11:20:17.118958 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:17.123651 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b44e6310-1302-4363-ae7d-5eca7b23c4e3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:17.386973 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5bd8ac89-d50e-40ba-b7a1-761f6ea08cee req-6c524e56-ce7f-419b-8858-2bace78653b7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.3136449 May 24 11:20:17.471022 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2fd7087-17cb-409f-9d31-c293469676bd req-8a5b52fd-5c18-4881-8617-c740c71fea62 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:17.472557 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d2fd7087-17cb-409f-9d31-c293469676bd req-8a5b52fd-5c18-4881-8617-c740c71fea62 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.5168221 May 24 11:20:17.729064 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:17.829420 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:17.983041 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d2fd7087-17cb-409f-9d31-c293469676bd req-c043e146-3efa-4151-a1bd-6a97f625e0f3 service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4774270 May 24 11:20:18.239336 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b44e6310-1302-4363-ae7d-5eca7b23c4e3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534788488', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535456750'] for subnet, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.266923 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-8fc246e0-3cab-4aa6-b5ab-b4965fd346aa admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:18.339700 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b44e6310-1302-4363-ae7d-5eca7b23c4e3 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=1ed78d41-8564-48a2-acc1-2ef21df88d9e,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:18.347417 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b44e6310-1302-4363-ae7d-5eca7b23c4e3 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.352665 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b44e6310-1302-4363-ae7d-5eca7b23c4e3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:18.437649 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-b44e6310-1302-4363-ae7d-5eca7b23c4e3 admin admin] 10.222.0.18 "DELETE /v2.0/subnets/1ed78d41-8564-48a2-acc1-2ef21df88d9e HTTP/1.1" status: 204 len: 168 time: 2.4000673 May 24 11:20:18.437987 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57688) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:18.440418 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d2fd7087-17cb-409f-9d31-c293469676bd req-2ea916a8-be5b-44c3-8fdb-aedd9e1eb186 service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3886259 May 24 11:20:18.494620 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 839b9282-2a46-4561-9d08-f49bad628dc7 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:20:18.514171 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Allocated IP 172.20.0.1 (7877e643-fe9e-4041-91fa-3a4f44279fd2/12ff35e8-9af8-4339-95c9-c62c90d8c6fd/9c46ec57-71f9-47d3-8520-7247c2991687) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:20:18.593585 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 839b9282-2a46-4561-9d08-f49bad628dc7 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:20:18.598555 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 839b9282-2a46-4561-9d08-f49bad628dc7 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:20:18.598929 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.641688 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.649364 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.662350 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.668502 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-991c1c0d-72e5-4390-8de4-0f13b8933fb9 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:18.711979 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25c93d91-261c-4ad3-bde1-eec4fc901d0c req-a8b59385-9f4d-46b0-a726-6e3f9ccc0365 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:18.713501 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-25c93d91-261c-4ad3-bde1-eec4fc901d0c req-a8b59385-9f4d-46b0-a726-6e3f9ccc0365 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1798470 May 24 11:20:18.720915 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.739250 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 839b9282-2a46-4561-9d08-f49bad628dc7, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=839b9282-2a46-4561-9d08-f49bad628dc7,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:20:18.743154 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.745147 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:18.819834 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=839b9282-2a46-4561-9d08-f49bad628dc7,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:18.823220 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'Port': ['ID=0cf9a14b-892a-4235-8708-1f72096a9a29,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:18.823581 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:18.838441 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.170s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:18.842223 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.859900 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['a21e7e1e-54f7-4844-b98d-f658bdb26a4c']}} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:18.870363 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-25c93d91-261c-4ad3-bde1-eec4fc901d0c req-3f29bdee-59be-492f-b76b-5c1b2c620d49 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1411257 May 24 11:20:18.874455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.876765 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'f3295bfe-6894-409e-b0cc-96311bd90247') {'for_net_delete': True} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:18.877220 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'f3295bfe-6894-409e-b0cc-96311bd90247') {} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:18.890466 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534853183'] for segment, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.909868 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.917813 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] Releasing vlan 514 on physical network physnet1 to pool {{(pid=91819) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:20:18.918368 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535155490'] for segment, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: Traceback (most recent call last): May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: timer() May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: cb(*args, **kw) May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: return func(*args, **kwargs) May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: self._notify() May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: self.callback(batched_events) May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: event.method(event) May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: return f(self, *a, **k) May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._get(url).json() May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: return self.request(url, 'GET', **kwargs) May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:20:18.937145 dv-cinder-784092-9-iscsi neutron-server[91819]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:20:18.939385 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:19.022669 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:19.027683 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-bc81c096-39a2-425f-b994-2dcf32cfd09d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:19.043656 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:19.045158 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-62dff87b-6f9f-4c44-92b3-78ef68f19d21 req-e14a980a-4408-48d6-812b-7c5988394de5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:19.046654 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-62dff87b-6f9f-4c44-92b3-78ef68f19d21 req-e14a980a-4408-48d6-812b-7c5988394de5 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=788050c389224d17b163f24bb67902bf&device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 1137 time: 0.4850922 May 24 11:20:19.049847 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] Pushing event deleted for resources: {'Network': ['ID=a21e7e1e-54f7-4844-b98d-f658bdb26a4c,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:19.051368 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Transition to ACTIVE for port object 9c46ec57-71f9-47d3-8520-7247c2991687 will not be triggered until provisioned by entity DHCP. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:20:19.055488 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:19.065367 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-ed6aa249-65ec-4d46-9de6-c01f1f512819 admin admin] 10.222.0.18 "DELETE /v2.0/networks/a21e7e1e-54f7-4844-b98d-f658bdb26a4c HTTP/1.1" status: 204 len: 168 time: 0.6231215 May 24 11:20:19.192348 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57698) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:19.208308 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.l3_db [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Floating IP 6caa87da-7c20-4c7c-a70d-c42930870a74 associated. External IP: 172.27.0.211, port: 8a7c6451-6355-44ad-a36a-f02ffcfe252c. May 24 11:20:19.259131 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-dcab0915-08eb-4557-af6d-7328a07237e2 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=7c9e587a510c4777ab57643ed31ea73f&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0652905 May 24 11:20:19.260242 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57702) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:19.359399 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-595b5867-ad8b-4f9e-81c2-4c109eda8d73 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:19.388724 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-595b5867-ad8b-4f9e-81c2-4c109eda8d73 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:19.448472 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-595b5867-ad8b-4f9e-81c2-4c109eda8d73 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:19.453341 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:19.468135 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-595b5867-ad8b-4f9e-81c2-4c109eda8d73 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=57604184-194b-41df-8615-c443c9e6dfa4,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:19.471559 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-595b5867-ad8b-4f9e-81c2-4c109eda8d73 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:19.475494 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:19.483490 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-595b5867-ad8b-4f9e-81c2-4c109eda8d73 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:19.486922 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-595b5867-ad8b-4f9e-81c2-4c109eda8d73 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/57604184-194b-41df-8615-c443c9e6dfa4 HTTP/1.1" status: 204 len: 168 time: 0.2227592 May 24 11:20:19.491176 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:19.547898 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-62dff87b-6f9f-4c44-92b3-78ef68f19d21 req-a7b052a5-10e3-42fc-b87c-66a98d8bc0c6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:19.549367 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-62dff87b-6f9f-4c44-92b3-78ef68f19d21 req-a7b052a5-10e3-42fc-b87c-66a98d8bc0c6 service neutron] 10.222.0.93 "GET /v2.0/networks?id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 HTTP/1.1" status: 200 len: 915 time: 0.4946532 May 24 11:20:19.645819 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Network 7877e643-fe9e-4041-91fa-3a4f44279fd2 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:20:19.657792 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:20:19.669740 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57714) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:19.670348 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:20:19.708471 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Pushing event updated for resources: {'Port': ['ID=9c46ec57-71f9-47d3-8520-7247c2991687,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:19.728914 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.238s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:19.772061 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-62dff87b-6f9f-4c44-92b3-78ef68f19d21 req-25b1b0c8-992e-47a5-a966-e1ca17eec9ad service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010 HTTP/1.1" status: 200 len: 1001 time: 0.2091703 May 24 11:20:19.777479 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-687570'] for router_interface, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:19.966129 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3880df7f-8735-4ad5-8097-dc6c136d9b9f admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:19.985724 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3880df7f-8735-4ad5-8097-dc6c136d9b9f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:19.990649 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-3880df7f-8735-4ad5-8097-dc6c136d9b9f admin admin] Usage tracker for resource:security_group_rule and tenant:3f7d405f7a6b45798cc8786b09780d20 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:19.991470 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-d723a261-3396-4a9c-a406-85245a76b2f2 None None] Provisioning for port 9c46ec57-71f9-47d3-8520-7247c2991687 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:20:19.994696 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-3880df7f-8735-4ad5-8097-dc6c136d9b9f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:19.997880 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-d723a261-3396-4a9c-a406-85245a76b2f2 None None] Provisioning complete for port 9c46ec57-71f9-47d3-8520-7247c2991687 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:20:19.998272 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d723a261-3396-4a9c-a406-85245a76b2f2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:20.001247 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3880df7f-8735-4ad5-8097-dc6c136d9b9f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:20.081882 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-d723a261-3396-4a9c-a406-85245a76b2f2 None None] Port 9c46ec57-71f9-47d3-8520-7247c2991687 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:20:20.123349 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3880df7f-8735-4ad5-8097-dc6c136d9b9f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:20.128009 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3880df7f-8735-4ad5-8097-dc6c136d9b9f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:20.142219 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e8cbc004-2c96-42ef-8886-b3899fc69fc1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:20.183123 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-3880df7f-8735-4ad5-8097-dc6c136d9b9f admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=3f7d405f7a6b45798cc8786b09780d20&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5110056 May 24 11:20:20.184111 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3880df7f-8735-4ad5-8097-dc6c136d9b9f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=616b5236-9dbb-43cb-94ce-e78f3ef72bad,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:20.187986 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3880df7f-8735-4ad5-8097-dc6c136d9b9f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:20.188473 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57734) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:20.252328 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-89b315fa-7ca4-4254-bc0c-fed062d227ee admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:20.255415 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-62dff87b-6f9f-4c44-92b3-78ef68f19d21 req-af47070b-dacf-48c1-addf-eea46485568b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:20.256951 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-62dff87b-6f9f-4c44-92b3-78ef68f19d21 req-af47070b-dacf-48c1-addf-eea46485568b service neutron] 10.222.0.93 "GET /v2.0/subnets?id=6a80c579-20d5-46c7-8c07-6438f1367afd HTTP/1.1" status: 200 len: 853 time: 0.3240681 May 24 11:20:20.284452 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-89b315fa-7ca4-4254-bc0c-fed062d227ee admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:20.343475 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-89b315fa-7ca4-4254-bc0c-fed062d227ee admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:20.350506 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-886b88a8-2f76-4190-837d-118df2ae4b33 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:20.363788 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-89b315fa-7ca4-4254-bc0c-fed062d227ee admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=616b5236-9dbb-43cb-94ce-e78f3ef72bad,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:20.378621 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-89b315fa-7ca4-4254-bc0c-fed062d227ee admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:20.381346 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-89b315fa-7ca4-4254-bc0c-fed062d227ee admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:20.386315 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-89b315fa-7ca4-4254-bc0c-fed062d227ee admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/616b5236-9dbb-43cb-94ce-e78f3ef72bad HTTP/1.1" status: 204 len: 168 time: 0.1964550 May 24 11:20:20.429836 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Router 81705dbf-841d-45b4-8b2d-afa9886a8e59 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:20:20.474032 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-093987eb-5f8f-4c19-b689-4479bc6be819 req-e21ab9e7-765f-49bf-ba63-ae5e33c50526 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:20.475328 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-093987eb-5f8f-4c19-b689-4479bc6be819 req-e21ab9e7-765f-49bf-ba63-ae5e33c50526 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.3698161 May 24 11:20:20.488936 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-62dff87b-6f9f-4c44-92b3-78ef68f19d21 req-f090f7b6-aba6-41de-b859-1a9bbce52176 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:20.490525 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-62dff87b-6f9f-4c44-92b3-78ef68f19d21 req-f090f7b6-aba6-41de-b859-1a9bbce52176 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.2221379 May 24 11:20:20.506773 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 839b9282-2a46-4561-9d08-f49bad628dc7 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:20:20.631637 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 839b9282-2a46-4561-9d08-f49bad628dc7 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:20:20.632600 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:20.741677 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:20.866231 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57754) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:20.885485 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e69db118-0bd4-4e49-b3b3-07301ea771ef admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:20.902449 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e69db118-0bd4-4e49-b3b3-07301ea771ef admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:20.904810 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-093987eb-5f8f-4c19-b689-4479bc6be819 req-b11bbd05-cac8-43e4-846d-8431260fb7b4 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.4207075 May 24 11:20:20.906784 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e69db118-0bd4-4e49-b3b3-07301ea771ef admin admin] Usage tracker for resource:security_group_rule and tenant:2f4bbb7e3626409fb40d3e5f98cea639 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:20.916254 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e69db118-0bd4-4e49-b3b3-07301ea771ef admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:20.923333 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e69db118-0bd4-4e49-b3b3-07301ea771ef admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:21.100997 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-62dff87b-6f9f-4c44-92b3-78ef68f19d21 req-57c042d0-eff7-497e-8d4b-97a0d624ba51 service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.5880694 May 24 11:20:21.109312 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:20:21.120819 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5ffae7e9-2401-4a04-8bd5-f4e3cd5cbfd5 None None] Sync routers for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:20:21.138216 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5ffae7e9-2401-4a04-8bd5-f4e3cd5cbfd5 None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:20:21.138845 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-5ffae7e9-2401-4a04-8bd5-f4e3cd5cbfd5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:19:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:21.173430 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e69db118-0bd4-4e49-b3b3-07301ea771ef admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:21.179842 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e69db118-0bd4-4e49-b3b3-07301ea771ef admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:21.189835 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-1fd58be1-e931-4478-b8c9-80d452d17260 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:21.238992 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:21.243765 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e69db118-0bd4-4e49-b3b3-07301ea771ef admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=2f4bbb7e3626409fb40d3e5f98cea639&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3764372 May 24 11:20:21.244740 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e69db118-0bd4-4e49-b3b3-07301ea771ef admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=29bf0f02-d46a-435e-8079-d3f8fbc620a4,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:21.260217 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e69db118-0bd4-4e49-b3b3-07301ea771ef admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:21.261287 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57794) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:21.373480 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-be2d3c36-2728-4b66-a96d-6997a9660bf2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:21.408870 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-be2d3c36-2728-4b66-a96d-6997a9660bf2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:21.455339 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-5ffae7e9-2401-4a04-8bd5-f4e3cd5cbfd5 None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:20:21.483283 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-62dff87b-6f9f-4c44-92b3-78ef68f19d21 req-c238aa52-eb28-4cda-820a-ae475d1f8e6e service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3716700 May 24 11:20:21.484792 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 996 time: 8.5106432 May 24 11:20:21.487794 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'tag': '8a7c6451-6355-44ad-a36a-f02ffcfe252c'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:20:21.524721 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:21.532654 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:21.565259 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-451f6344-ffaa-4783-aada-803ae270040a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:21.579271 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d08610b-38ac-4e11-9a50-08ae59b99851 req-e07f5ada-63ca-4f7b-99ca-c2244d5c8630 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:21.580917 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d08610b-38ac-4e11-9a50-08ae59b99851 req-e07f5ada-63ca-4f7b-99ca-c2244d5c8630 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=b303f83a84794e5ba6a1bab9c76b429c&device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 1099 time: 0.1637933 May 24 11:20:21.617861 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e7526abc-f423-4b6b-8393-a98460566a53 req-e47f6091-8242-4f07-86cc-569d46308303 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:21.619397 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e7526abc-f423-4b6b-8393-a98460566a53 req-e47f6091-8242-4f07-86cc-569d46308303 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.3460684 May 24 11:20:21.648364 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'tag': '8a7c6451-6355-44ad-a36a-f02ffcfe252c', 'status': 'completed', 'code': 200} May 24 11:20:21.656601 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-be2d3c36-2728-4b66-a96d-6997a9660bf2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:21.743887 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Pushing event updated for resources: {'Port': ['ID=9c46ec57-71f9-47d3-8520-7247c2991687,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:21.747857 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.182s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:21.751526 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:21.754330 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-533332', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-687487'] for router_interface, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:21.766883 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-be2d3c36-2728-4b66-a96d-6997a9660bf2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=29bf0f02-d46a-435e-8079-d3f8fbc620a4,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:21.772022 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-be2d3c36-2728-4b66-a96d-6997a9660bf2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:21.781463 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-be2d3c36-2728-4b66-a96d-6997a9660bf2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:21.782752 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-be2d3c36-2728-4b66-a96d-6997a9660bf2 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/29bf0f02-d46a-435e-8079-d3f8fbc620a4 HTTP/1.1" status: 204 len: 168 time: 0.5209610 May 24 11:20:21.801615 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e7526abc-f423-4b6b-8393-a98460566a53 req-0dcda1a1-16ab-4287-b56e-e7b489632107 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1522992 May 24 11:20:21.859664 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d08610b-38ac-4e11-9a50-08ae59b99851 req-f3ada87d-cfc2-4f6e-adca-c9155348d0a5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:21.861463 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d08610b-38ac-4e11-9a50-08ae59b99851 req-f3ada87d-cfc2-4f6e-adca-c9155348d0a5 service neutron] 10.222.0.93 "GET /v2.0/networks?id=1d92641a-d8f9-4ed6-9abd-f464bb26811d HTTP/1.1" status: 200 len: 912 time: 0.2728431 May 24 11:20:21.956583 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95a6d6d6-7e58-4c06-b2d2-5a984c9129df req-c681a031-1b14-4c9f-a833-14fb08160459 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:21.958382 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-95a6d6d6-7e58-4c06-b2d2-5a984c9129df req-c681a031-1b14-4c9f-a833-14fb08160459 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 1098 time: 0.2687194 May 24 11:20:22.015367 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d08610b-38ac-4e11-9a50-08ae59b99851 req-f2f5c94d-9e46-4d93-9c64-ffe7e19bf087 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=eab44150-44f7-4194-9b76-436d421794e8 HTTP/1.1" status: 200 len: 1001 time: 0.1471996 May 24 11:20:22.029944 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:22.088231 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Network 7877e643-fe9e-4041-91fa-3a4f44279fd2 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:20:22.091850 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:20:22.102410 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d08610b-38ac-4e11-9a50-08ae59b99851 req-d5c11337-ba01-4c86-92d3-408dd7e9016b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:22.104059 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d08610b-38ac-4e11-9a50-08ae59b99851 req-d5c11337-ba01-4c86-92d3-408dd7e9016b service neutron] 10.222.0.93 "GET /v2.0/subnets?id=2fc4be2e-875c-4d10-b857-a48f90c53cf2 HTTP/1.1" status: 200 len: 850 time: 0.0811443 May 24 11:20:22.175422 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-5ffae7e9-2401-4a04-8bd5-f4e3cd5cbfd5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '6caa87da-7c20-4c7c-a70d-c42930870a74', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.211', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': '8a7c6451-6355-44ad-a36a-f02ffcfe252c', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 449, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:f3:93:af', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:20:16Z', 'updated_at': '2021-05-24T11:20:16Z', 'revision_number': 0, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:22.181176 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5ffae7e9-2401-4a04-8bd5-f4e3cd5cbfd5 None None] Checking router: 81705dbf-841d-45b4-8b2d-afa9886a8e59 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:20:22.191633 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5ffae7e9-2401-4a04-8bd5-f4e3cd5cbfd5 None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:20:22.192255 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-5ffae7e9-2401-4a04-8bd5-f4e3cd5cbfd5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:19:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:22.211257 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d08610b-38ac-4e11-9a50-08ae59b99851 req-9086ec27-8774-49bf-adb6-53530bf3f975 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:22.212934 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d08610b-38ac-4e11-9a50-08ae59b99851 req-9086ec27-8774-49bf-adb6-53530bf3f975 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=1d92641a-d8f9-4ed6-9abd-f464bb26811d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1021843 May 24 11:20:22.412209 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d08610b-38ac-4e11-9a50-08ae59b99851 req-f10debea-a9ac-4f09-a312-a6a3221ba456 service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1892517 May 24 11:20:22.434140 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Router c2ffed60-7ad5-4215-9e01-680707fb98d6 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:20:22.435746 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-7771c611-f0d5-4519-a76e-3a84fbaf1106 None None] Provisioning complete for port 9c46ec57-71f9-47d3-8520-7247c2991687 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:20:22.437157 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-7771c611-f0d5-4519-a76e-3a84fbaf1106 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:22.524359 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-7771c611-f0d5-4519-a76e-3a84fbaf1106 None None] Port 9c46ec57-71f9-47d3-8520-7247c2991687 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:20:22.563054 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-5ffae7e9-2401-4a04-8bd5-f4e3cd5cbfd5 None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:20:22.624610 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d08610b-38ac-4e11-9a50-08ae59b99851 req-e48a8471-0a73-48ea-a308-1670f7d93abe service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2021170 May 24 11:20:22.639698 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95a6d6d6-7e58-4c06-b2d2-5a984c9129df req-3ad04b03-e0cd-4987-b147-eb4e125c625a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:22.641720 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-95a6d6d6-7e58-4c06-b2d2-5a984c9129df req-3ad04b03-e0cd-4987-b147-eb4e125c625a service neutron] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 912 time: 0.6711578 May 24 11:20:22.800308 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-95a6d6d6-7e58-4c06-b2d2-5a984c9129df req-1bf8483e-ee62-4eb3-8b97-2eb06f19dc21 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=8a7c6451-6355-44ad-a36a-f02ffcfe252c HTTP/1.1" status: 200 len: 999 time: 0.1358190 May 24 11:20:23.034397 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-5ffae7e9-2401-4a04-8bd5-f4e3cd5cbfd5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '6caa87da-7c20-4c7c-a70d-c42930870a74', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.211', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': '8a7c6451-6355-44ad-a36a-f02ffcfe252c', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 449, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:f3:93:af', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:20:16Z', 'updated_at': '2021-05-24T11:20:16Z', 'revision_number': 0, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:23.041633 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5ffae7e9-2401-4a04-8bd5-f4e3cd5cbfd5 None None] The sync data for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi is: [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '6caa87da-7c20-4c7c-a70d-c42930870a74', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.211', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': '8a7c6451-6355-44ad-a36a-f02ffcfe252c', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 449, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:f3:93:af', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:20:16Z', 'updated_at': '2021-05-24T11:20:16Z', 'revision_number': 0, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:20:23.076115 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:20:23.091465 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] 10.222.0.18 "PUT /v2.0/routers/c2ffed60-7ad5-4215-9e01-680707fb98d6/add_router_interface HTTP/1.1" status: 200 len: 503 time: 8.2412221 May 24 11:20:23.096726 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Sync routers for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:20:23.115216 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-574f5e48-775c-4099-9a39-5d427268c6da req-d2966be0-7a2d-4924-a156-15a25d446a03 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:23.127138 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-574f5e48-775c-4099-9a39-5d427268c6da req-d2966be0-7a2d-4924-a156-15a25d446a03 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1434586 May 24 11:20:23.127544 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95a6d6d6-7e58-4c06-b2d2-5a984c9129df req-9c627d57-7575-41da-987b-1cde6009acaf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:23.129422 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-95a6d6d6-7e58-4c06-b2d2-5a984c9129df req-9c627d57-7575-41da-987b-1cde6009acaf service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.3112400 May 24 11:20:23.133344 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:20:23.134875 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:19:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:23.229777 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-574f5e48-775c-4099-9a39-5d427268c6da req-e03d782e-f0a5-4030-8c81-7cdd4f678440 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1014404 May 24 11:20:23.306404 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95a6d6d6-7e58-4c06-b2d2-5a984c9129df req-3af9391f-dab3-440e-adbb-ebfbc3840e08 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:23.308419 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-95a6d6d6-7e58-4c06-b2d2-5a984c9129df req-3af9391f-dab3-440e-adbb-ebfbc3840e08 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1702461 May 24 11:20:23.424034 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9aa2dc9c-54e2-48bf-a1c1-ac2a9cf3ae03 None None] New status for floating IP 6caa87da-7c20-4c7c-a70d-c42930870a74: ACTIVE {{(pid=91822) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:20:23.476658 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:20:23.621934 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-95a6d6d6-7e58-4c06-b2d2-5a984c9129df req-d415d6a2-1cad-4d63-80ba-e4ef6cf22391 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3042827 May 24 11:20:23.723383 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-254cac02-525a-47b2-b849-069257d5de60 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:23.896578 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-95a6d6d6-7e58-4c06-b2d2-5a984c9129df req-87aad747-a03a-41d5-80f1-887dd0549966 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2663002 May 24 11:20:23.927327 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:21Z', 'revision_number': 2, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:23.933388 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Checking router: c2ffed60-7ad5-4215-9e01-680707fb98d6 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:20:23.933755 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Updating router c2ffed60-7ad5-4215-9e01-680707fb98d6 port 9c46ec57-71f9-47d3-8520-7247c2991687 binding host dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:20:24.011834 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.244808 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57870) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:24.251198 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Request body: {'security_group': {'name': 'tempest-securitygroup--750940266', 'description': 'tempest-description--655398584'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:24.259780 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:24.265632 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Usage tracker for resource:security_group and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:24.268731 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:24.278338 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:24.293191 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.325299 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:24.330018 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Usage tracker for resource:security_group_rule and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:24.334239 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:24.344939 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:24.435748 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Notify callbacks [] for port, precommit_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.452040 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Transition to ACTIVE for port object 9c46ec57-71f9-47d3-8520-7247c2991687 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:20:24.472697 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.477808 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:24.522175 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.536055 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.558707 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9650459b-3c7e-42d5-ae92-2b4e4cdb032e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:24.574089 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.581730 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1778 time: 0.3298521 May 24 11:20:24.582592 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 57882) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:24.584328 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25fd90e2-d370-44d3-a561-4c4b68219cea req-1ad795ce-0e6b-414b-b706-997f0a62ca57 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:24.586308 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-25fd90e2-d370-44d3-a561-4c4b68219cea req-1ad795ce-0e6b-414b-b706-997f0a62ca57 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1859813 May 24 11:20:24.589378 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-e03bcb69-fd98-442e-b642-c6a6fc01929a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Request body: {'security_group_rule': {'security_group_id': 'eaab864c-81a2-4598-945c-15413cce7a89', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:24.598099 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-e03bcb69-fd98-442e-b642-c6a6fc01929a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:24.618372 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-e03bcb69-fd98-442e-b642-c6a6fc01929a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Usage tracker for resource:security_group_rule and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:24.624891 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-e03bcb69-fd98-442e-b642-c6a6fc01929a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:24.637292 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-e03bcb69-fd98-442e-b642-c6a6fc01929a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:24.647639 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event updated for resources: {'SecurityGroup': ['ID=eaab864c-81a2-4598-945c-15413cce7a89,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:24.657027 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:24.663800 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-46d24114-42cf-4dc3-ad05-93eb4a0ec306 None None] Pushing event updated for resources: {'Port': ['ID=9c46ec57-71f9-47d3-8520-7247c2991687,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:24.667735 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e03bcb69-fd98-442e-b642-c6a6fc01929a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.668491 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-25fd90e2-d370-44d3-a561-4c4b68219cea req-7873c168-0dde-4152-9a9d-31814ca139b2 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0673075 May 24 11:20:24.670915 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-46d24114-42cf-4dc3-ad05-93eb4a0ec306 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.193s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:24.787234 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e03bcb69-fd98-442e-b642-c6a6fc01929a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.793613 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e03bcb69-fd98-442e-b642-c6a6fc01929a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.798028 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Attempting to bind port 9c46ec57-71f9-47d3-8520-7247c2991687 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:20:24.798463 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Attempting to bind port 9c46ec57-71f9-47d3-8520-7247c2991687 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:20:24.798846 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Attempting to bind port 9c46ec57-71f9-47d3-8520-7247c2991687 on network 7877e643-fe9e-4041-91fa-3a4f44279fd2 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:20:24.805605 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 19, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:20:24.806042 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Checking segment: {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:20:24.806461 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.807595 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Bound using segment: {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:20:24.807896 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:24.808841 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Bound port: 9c46ec57-71f9-47d3-8520-7247c2991687, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:20:24.808841 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.835358 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e03bcb69-fd98-442e-b642-c6a6fc01929a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=499cfe56-56f8-4889-8da4-1e32334bb4f9,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:24.839922 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e03bcb69-fd98-442e-b642-c6a6fc01929a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:24.844323 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e03bcb69-fd98-442e-b642-c6a6fc01929a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.845646 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-e03bcb69-fd98-442e-b642-c6a6fc01929a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.2615023 May 24 11:20:24.849311 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57892) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:24.855068 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-80e3d592-b40e-4250-8fb2-9d450306e267 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Request body: {'security_group_rule': {'security_group_id': 'eaab864c-81a2-4598-945c-15413cce7a89', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:24.867082 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-80e3d592-b40e-4250-8fb2-9d450306e267 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:24.874546 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-80e3d592-b40e-4250-8fb2-9d450306e267 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Usage tracker for resource:security_group_rule and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:24.877718 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-80e3d592-b40e-4250-8fb2-9d450306e267 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:24.891114 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-80e3d592-b40e-4250-8fb2-9d450306e267 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:24.921398 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-80e3d592-b40e-4250-8fb2-9d450306e267 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.937909 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] For port 9c46ec57-71f9-47d3-8520-7247c2991687, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:20:24.942721 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] For port 9c46ec57-71f9-47d3-8520-7247c2991687, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=9c46ec57-71f9-47d3-8520-7247c2991687,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:20:24.960963 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:24.966133 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:24.996670 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-80e3d592-b40e-4250-8fb2-9d450306e267 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:25.004748 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-80e3d592-b40e-4250-8fb2-9d450306e267 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:25.015444 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-cf8ee319-e7db-4496-a101-42a8acbfcd4c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:25.045892 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-80e3d592-b40e-4250-8fb2-9d450306e267 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=6ad0268a-805b-4a3e-baac-b998310e7daf,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:25.052851 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:20:25.053902 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:19:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:25.054523 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-80e3d592-b40e-4250-8fb2-9d450306e267 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:25.062505 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-80e3d592-b40e-4250-8fb2-9d450306e267 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:25.063968 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-80e3d592-b40e-4250-8fb2-9d450306e267 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.2134609 May 24 11:20:25.067200 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57900) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:25.073354 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Request body: {'floatingip': {'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:25.080365 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:25.085122 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Usage tracker for resource:floatingip and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:25.088791 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:25.099934 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:25.120665 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:25.251635 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-46d24114-42cf-4dc3-ad05-93eb4a0ec306 None None] Pushing event updated for resources: {'Port': ['ID=9c46ec57-71f9-47d3-8520-7247c2991687,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:25.266024 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-46d24114-42cf-4dc3-ad05-93eb4a0ec306 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.300s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:25.353779 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:20:25.355706 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:25.517500 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Allocated IP 172.27.0.70 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/43cda913-ddcb-408c-bbe1-fb1518ca5b28) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:20:25.771313 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:25.784718 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:25.814203 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:25.819129 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:25.829422 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:24Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:25.836977 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c35fc997-34f6-40ed-a031-308a226e2357 None None] The sync data for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi is: [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:24Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:20:25.889932 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:20:25.891094 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:20:25.893842 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:20:25.902511 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-40b5e2bf-0523-4d52-a05f-6ca11bba9cb4 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:25.948618 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event updated for resources: {'Port': ['ID=43cda913-ddcb-408c-bbe1-fb1518ca5b28,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:25.965447 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.140s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:26.029730 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:26.099304 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:26.329923 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:26.336522 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:26.341948 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-936d7e03-dac2-44f7-8d01-61e0e1239cb8 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:26.454049 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event updated for resources: {'Port': ['ID=43cda913-ddcb-408c-bbe1-fb1518ca5b28,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:26.459245 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:26.466396 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:26.487364 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 9c46ec57-71f9-47d3-8520-7247c2991687 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:20:26.491740 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:26.505911 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 692 time: 1.4330497 May 24 11:20:26.578659 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 9c46ec57-71f9-47d3-8520-7247c2991687 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:20:26.586436 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 9c46ec57-71f9-47d3-8520-7247c2991687 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:20:26.586930 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:26.712058 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:26.750659 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 9c46ec57-71f9-47d3-8520-7247c2991687, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=9c46ec57-71f9-47d3-8520-7247c2991687,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:20:26.758161 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:26.760019 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:26.775326 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-9884096f-49fc-441f-b9a1-02ae492a2530 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Synchronizing usage tracker for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:security_group {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:20:26.796846 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-9884096f-49fc-441f-b9a1-02ae492a2530 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Unset dirty status for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:security_group {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:20:26.798057 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-9884096f-49fc-441f-b9a1-02ae492a2530 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?name=tempest-securitygroup--750940266&fields=id&tenant_id=05ec02ff35d2402e8d7bf064752e67ea HTTP/1.1" status: 200 len: 267 time: 0.0692661 May 24 11:20:26.834010 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=9c46ec57-71f9-47d3-8520-7247c2991687,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:26.837417 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:26.971994 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:26.989026 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-33d1b279-5545-4b21-9b12-2ea5ad9ae353 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:26.989811 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-33d1b279-5545-4b21-9b12-2ea5ad9ae353 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Synchronizing usage tracker for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:network {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:20:27.019997 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-33d1b279-5545-4b21-9b12-2ea5ad9ae353 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Unset dirty status for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:network {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:20:27.021334 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-33d1b279-5545-4b21-9b12-2ea5ad9ae353 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 816 time: 0.2146869 May 24 11:20:27.036701 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-c5a62036-c519-4050-8112-6e677088389d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/quotas/05ec02ff35d2402e8d7bf064752e67ea HTTP/1.1" status: 200 len: 375 time: 0.0098820 May 24 11:20:27.135479 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-5eb602fc-c38b-4451-b300-7faf2b07e2ee tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Synchronizing usage tracker for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:20:27.155166 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-5eb602fc-c38b-4451-b300-7faf2b07e2ee tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Unset dirty status for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:20:27.156536 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-5eb602fc-c38b-4451-b300-7faf2b07e2ee tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&fields=id HTTP/1.1" status: 200 len: 348 time: 0.1120796 May 24 11:20:27.309272 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-4f7bad7d-498f-4c3d-8f06-cdd3c588a601 service neutron] 10.222.0.18 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1395590 May 24 11:20:27.457710 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-c62c43fc-a0e9-4082-9731-cf493eb054db service neutron] 10.222.0.18 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1425672 May 24 11:20:27.669393 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0431c1a1-9c95-4578-bba3-a4741d0af126 req-499a73b4-b8b9-474b-a684-ead0f93aa753 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 210 time: 0.0494010 May 24 11:20:28.436855 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 9c46ec57-71f9-47d3-8520-7247c2991687 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:20:28.512522 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 9c46ec57-71f9-47d3-8520-7247c2991687 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:20:28.512894 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:28.864065 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1c8f2984-237d-49c2-9215-0cea0e153dfe req-c2969043-1135-4708-aec4-a0d47411ca08 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 210 time: 0.0609944 May 24 11:20:28.909039 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-aa4329fd-3032-425b-88bb-211a0362fc64 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:28.911449 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-aa4329fd-3032-425b-88bb-211a0362fc64 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.93 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 816 time: 0.2344227 May 24 11:20:28.924094 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:28.969441 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-f9ac9d6a-b598-401d-9b26-688b5c866000 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.93 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=05ec02ff35d2402e8d7bf064752e67ea HTTP/1.1" status: 200 len: 379 time: 0.0489037 May 24 11:20:28.978709 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Request body: {'port': {'device_id': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'admin_state_up': True, 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'security_groups': ['eaab864c-81a2-4598-945c-15413cce7a89']}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:29.124683 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:29.135139 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:29.148424 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Made reservation on behalf of 05ec02ff35d2402e8d7bf064752e67ea for: {'port': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:20:29.152066 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:29.226222 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57990) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:29.333500 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Allocated IP 172.20.0.12 (7877e643-fe9e-4041-91fa-3a4f44279fd2/12ff35e8-9af8-4339-95c9-c62c90d8c6fd/9376988c-0749-43f3-a506-c8c29a15cbcf) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:20:29.350513 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-612a357f-460b-4cb8-8ecd-f454527e9970 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:29.365311 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-612a357f-460b-4cb8-8ecd-f454527e9970 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:29.369240 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-612a357f-460b-4cb8-8ecd-f454527e9970 admin admin] Usage tracker for resource:security_group_rule and tenant:e1cc9fe80bb040ac91ac86240f6f0fcb is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:29.372054 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-612a357f-460b-4cb8-8ecd-f454527e9970 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:29.377793 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-612a357f-460b-4cb8-8ecd-f454527e9970 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:29.467253 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-612a357f-460b-4cb8-8ecd-f454527e9970 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:29.471600 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-612a357f-460b-4cb8-8ecd-f454527e9970 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:29.479367 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8b73facb-c480-46c4-b1e2-90eecf3c5210 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:29.525997 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-612a357f-460b-4cb8-8ecd-f454527e9970 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=e1cc9fe80bb040ac91ac86240f6f0fcb&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2986529 May 24 11:20:29.527336 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 57996) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:29.527640 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-612a357f-460b-4cb8-8ecd-f454527e9970 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7e4f4be7-1441-4af1-8579-93e5a3d4e721,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:29.532285 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-612a357f-460b-4cb8-8ecd-f454527e9970 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:29.604788 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d036c90a-ef32-40b3-a0b7-a1e1feb27369 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:29.631473 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d036c90a-ef32-40b3-a0b7-a1e1feb27369 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:29.647968 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:29.667100 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Transition to ACTIVE for port object 9376988c-0749-43f3-a506-c8c29a15cbcf will not be triggered until provisioned by entity DHCP. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:20:29.696494 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:29.697251 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d036c90a-ef32-40b3-a0b7-a1e1feb27369 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:29.701468 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-53b9b4ef-127a-4276-aa4a-6a101b874264 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:29.702388 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-599fee59-fdea-4f91-897d-17a4c47e9589 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:29.713560 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d036c90a-ef32-40b3-a0b7-a1e1feb27369 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7e4f4be7-1441-4af1-8579-93e5a3d4e721,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:29.716990 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d036c90a-ef32-40b3-a0b7-a1e1feb27369 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:29.720617 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d036c90a-ef32-40b3-a0b7-a1e1feb27369 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:29.721838 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-d036c90a-ef32-40b3-a0b7-a1e1feb27369 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/7e4f4be7-1441-4af1-8579-93e5a3d4e721 HTTP/1.1" status: 204 len: 168 time: 0.1913106 May 24 11:20:29.767690 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Network 7877e643-fe9e-4041-91fa-3a4f44279fd2 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:20:29.771547 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:20:29.775519 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:20:29.819408 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event updated for resources: {'Port': ['ID=9376988c-0749-43f3-a506-c8c29a15cbcf,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:29.825161 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.123s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:29.855053 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:29.855053 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:29.888666 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 0.9135575 May 24 11:20:29.902975 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Request body: {'port': {'device_id': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:30.024196 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.052782 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-6e99b2eb-17f8-4eaf-a8b2-9f8f4a55957f None None] Provisioning for port 9376988c-0749-43f3-a506-c8c29a15cbcf completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:20:30.057977 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-6e99b2eb-17f8-4eaf-a8b2-9f8f4a55957f None None] Provisioning complete for port 9376988c-0749-43f3-a506-c8c29a15cbcf triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:20:30.058500 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-6e99b2eb-17f8-4eaf-a8b2-9f8f4a55957f None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.059607 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58008) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:30.102877 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-41f656a5-9532-4de0-9691-bfe74c7625fb admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.121676 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-6e99b2eb-17f8-4eaf-a8b2-9f8f4a55957f None None] Port 9376988c-0749-43f3-a506-c8c29a15cbcf cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:20:30.126931 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-41f656a5-9532-4de0-9691-bfe74c7625fb admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:30.134476 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-41f656a5-9532-4de0-9691-bfe74c7625fb admin admin] Usage tracker for resource:security_group_rule and tenant:24510b4a40884b758d49eda06bb75c30 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:30.137910 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-41f656a5-9532-4de0-9691-bfe74c7625fb admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:30.145987 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-41f656a5-9532-4de0-9691-bfe74c7625fb admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:30.165168 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d7d54fe-c62f-40cd-8b37-40d14f7b1b19 req-5df7b0a7-6df0-41ce-8130-671fb0dff738 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:30.166679 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7d7d54fe-c62f-40cd-8b37-40d14f7b1b19 req-5df7b0a7-6df0-41ce-8130-671fb0dff738 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 932 time: 0.1439805 May 24 11:20:30.228833 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-7d7d54fe-c62f-40cd-8b37-40d14f7b1b19 req-64fd5134-e111-43ce-8e6f-a608e792aace tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Synchronizing usage tracker for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:20:30.274277 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-7d7d54fe-c62f-40cd-8b37-40d14f7b1b19 req-64fd5134-e111-43ce-8e6f-a608e792aace tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Unset dirty status for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:20:30.275427 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7d7d54fe-c62f-40cd-8b37-40d14f7b1b19 req-64fd5134-e111-43ce-8e6f-a608e792aace tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1006329 May 24 11:20:30.291750 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-41f656a5-9532-4de0-9691-bfe74c7625fb admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.304541 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-41f656a5-9532-4de0-9691-bfe74c7625fb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.313454 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-dbc6a59d-f3e9-4615-b4b3-7a30e972699f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:30.356266 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-41f656a5-9532-4de0-9691-bfe74c7625fb admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=24510b4a40884b758d49eda06bb75c30&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2910726 May 24 11:20:30.358412 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-41f656a5-9532-4de0-9691-bfe74c7625fb admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=851cd2a0-1ad7-4c8b-9b08-a7b47fa72e65,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:30.360866 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58012) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:30.363922 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-41f656a5-9532-4de0-9691-bfe74c7625fb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:30.407963 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 9376988c-0749-43f3-a506-c8c29a15cbcf {{(pid=91820) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:20:30.416975 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.426829 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3c04653b-3f7e-4db1-8703-df6c4b9145ae admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.431887 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Transition to ACTIVE for port object 9376988c-0749-43f3-a506-c8c29a15cbcf will not be triggered until provisioned by entity L2. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:20:30.437818 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.442673 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:30.455199 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3c04653b-3f7e-4db1-8703-df6c4b9145ae admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.515213 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3c04653b-3f7e-4db1-8703-df6c4b9145ae admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.519938 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-19c0d399-ca3a-4567-897f-94329d013554 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:30.540763 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3c04653b-3f7e-4db1-8703-df6c4b9145ae admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=851cd2a0-1ad7-4c8b-9b08-a7b47fa72e65,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:30.551107 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3c04653b-3f7e-4db1-8703-df6c4b9145ae admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:30.551107 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3c04653b-3f7e-4db1-8703-df6c4b9145ae admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.551107 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-3c04653b-3f7e-4db1-8703-df6c4b9145ae admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/851cd2a0-1ad7-4c8b-9b08-a7b47fa72e65 HTTP/1.1" status: 204 len: 168 time: 0.1869621 May 24 11:20:30.640795 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Pushing event updated for resources: {'Port': ['ID=9376988c-0749-43f3-a506-c8c29a15cbcf,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:30.644590 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.202s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:30.729138 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Attempting to bind port 9376988c-0749-43f3-a506-c8c29a15cbcf on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:20:30.729478 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Attempting to bind port 9376988c-0749-43f3-a506-c8c29a15cbcf by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:20:30.729811 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Attempting to bind port 9376988c-0749-43f3-a506-c8c29a15cbcf on network 7877e643-fe9e-4041-91fa-3a4f44279fd2 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:20:30.734661 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Port 9376988c-0749-43f3-a506-c8c29a15cbcf on network 7877e643-fe9e-4041-91fa-3a4f44279fd2 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:20:30.734988 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Attempting to bind port 9376988c-0749-43f3-a506-c8c29a15cbcf on network 7877e643-fe9e-4041-91fa-3a4f44279fd2 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:20:30.740736 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 20, 22), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:20:30.741151 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Checking segment: {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:20:30.741508 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Bound using segment: {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:20:30.741943 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Bound port: 9376988c-0749-43f3-a506-c8c29a15cbcf, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'}}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:20:30.742291 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.854074 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, cleared binding levels {{(pid=91820) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:20:30.858275 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=9376988c-0749-43f3-a506-c8c29a15cbcf,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91820) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:20:30.862504 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.867338 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ab281a4d-12b2-42ed-852f-b3f82a6de445 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:30.929417 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:30.930028 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:30.931379 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] 10.222.0.93 "PUT /v2.0/ports/9376988c-0749-43f3-a506-c8c29a15cbcf HTTP/1.1" status: 200 len: 1094 time: 1.0338917 May 24 11:20:30.931886 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:20:30.967756 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Pushing event updated for resources: {'Port': ['ID=9376988c-0749-43f3-a506-c8c29a15cbcf,revision_number=3']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:30.971830 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:30.982543 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58024) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:31.010884 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a7953674-524d-4bf1-8acb-b32390f11b0e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:31.024098 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'status': 'completed', 'code': 200} May 24 11:20:31.028317 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-a7953674-524d-4bf1-8acb-b32390f11b0e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:31.032518 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-a7953674-524d-4bf1-8acb-b32390f11b0e admin admin] Usage tracker for resource:security_group_rule and tenant:eb212c268b4f42b1a17ad2bb1b8082a7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:31.037668 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-a7953674-524d-4bf1-8acb-b32390f11b0e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:31.043973 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-a7953674-524d-4bf1-8acb-b32390f11b0e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:31.066975 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-6167266a-9a5b-432a-8995-321f1d4eccbd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:31.068494 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-6167266a-9a5b-432a-8995-321f1d4eccbd service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 1097 time: 0.0805247 May 24 11:20:31.145056 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a7953674-524d-4bf1-8acb-b32390f11b0e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:31.149977 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a7953674-524d-4bf1-8acb-b32390f11b0e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:31.153080 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-7d3748dc-5333-4cf2-80d8-0b5e31c83056 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=9376988c-0749-43f3-a506-c8c29a15cbcf HTTP/1.1" status: 200 len: 217 time: 0.0717125 May 24 11:20:31.160823 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-14d9cbcb-a311-4e25-9319-c24463b24ad8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:31.205854 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-a7953674-524d-4bf1-8acb-b32390f11b0e admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=eb212c268b4f42b1a17ad2bb1b8082a7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2219367 May 24 11:20:31.207540 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a7953674-524d-4bf1-8acb-b32390f11b0e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=73f37498-a9b2-4bb8-b8e4-e6cca7ef0792,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:31.209542 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 58030) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:31.210792 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a7953674-524d-4bf1-8acb-b32390f11b0e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:31.244945 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-91a981a2-2483-4e27-b1c2-7c1d495aa38e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:31.246297 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-91a981a2-2483-4e27-b1c2-7c1d495aa38e service neutron] 10.222.0.93 "GET /v2.0/subnets?id=12ff35e8-9af8-4339-95c9-c62c90d8c6fd HTTP/1.1" status: 200 len: 855 time: 0.0847101 May 24 11:20:31.330864 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0cb4cb40-4044-4268-8674-89e7f450fa67 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:31.367176 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0cb4cb40-4044-4268-8674-89e7f450fa67 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:31.414466 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-55f0f739-ca83-43ae-859f-eb64d9840205 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:31.415967 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-55f0f739-ca83-43ae-859f-eb64d9840205 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=7877e643-fe9e-4041-91fa-3a4f44279fd2&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1449702 May 24 11:20:31.490920 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c2b881e8-7a81-4e11-83cd-80009fc1b4d6 req-995ced25-42f0-4f74-95b0-540953aa7b8a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:31.492512 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c2b881e8-7a81-4e11-83cd-80009fc1b4d6 req-995ced25-42f0-4f74-95b0-540953aa7b8a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.1045697 May 24 11:20:31.507220 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0cb4cb40-4044-4268-8674-89e7f450fa67 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:31.513172 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:31.530875 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0cb4cb40-4044-4268-8674-89e7f450fa67 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=73f37498-a9b2-4bb8-b8e4-e6cca7ef0792,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:31.537556 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c2b881e8-7a81-4e11-83cd-80009fc1b4d6 req-cd685fb1-19e9-43ac-99d9-4d2152e98df1 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0387189 May 24 11:20:31.537697 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0cb4cb40-4044-4268-8674-89e7f450fa67 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:31.541417 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0cb4cb40-4044-4268-8674-89e7f450fa67 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:31.544995 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-0cb4cb40-4044-4268-8674-89e7f450fa67 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/73f37498-a9b2-4bb8-b8e4-e6cca7ef0792 HTTP/1.1" status: 204 len: 168 time: 0.3316035 May 24 11:20:31.597815 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-dc6012fb-607e-47ed-afce-5ea34b602232 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1745362 May 24 11:20:31.844972 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5875b6a2-73ec-4ebc-915b-6cc0837a136e req-7fea177d-d656-478b-aa93-d61dcc56b776 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2397449 May 24 11:20:31.980173 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ceb423dd-e7a0-434b-839d-7c4ebd77bfc1 req-e02cc46b-1abb-4fe7-bf03-eadc2828b717 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:31.981862 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ceb423dd-e7a0-434b-839d-7c4ebd77bfc1 req-e02cc46b-1abb-4fe7-bf03-eadc2828b717 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 1097 time: 0.0826750 May 24 11:20:32.121962 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ceb423dd-e7a0-434b-839d-7c4ebd77bfc1 req-a3076f47-a710-41f6-95d1-65ddf719fcb2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:32.123412 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ceb423dd-e7a0-434b-839d-7c4ebd77bfc1 req-a3076f47-a710-41f6-95d1-65ddf719fcb2 service neutron] 10.222.0.93 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 917 time: 0.1343594 May 24 11:20:32.196253 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ceb423dd-e7a0-434b-839d-7c4ebd77bfc1 req-a1346941-d140-46cc-b547-25c062971f53 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=9376988c-0749-43f3-a506-c8c29a15cbcf HTTP/1.1" status: 200 len: 217 time: 0.0627675 May 24 11:20:32.277676 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ceb423dd-e7a0-434b-839d-7c4ebd77bfc1 req-7da34a90-02e5-4fe3-8411-513e9d6043a9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:32.279126 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ceb423dd-e7a0-434b-839d-7c4ebd77bfc1 req-7da34a90-02e5-4fe3-8411-513e9d6043a9 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=12ff35e8-9af8-4339-95c9-c62c90d8c6fd HTTP/1.1" status: 200 len: 855 time: 0.0629632 May 24 11:20:32.361287 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ceb423dd-e7a0-434b-839d-7c4ebd77bfc1 req-7a8e86c4-9f53-4251-863c-b64042d844cf service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:32.362975 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ceb423dd-e7a0-434b-839d-7c4ebd77bfc1 req-7a8e86c4-9f53-4251-863c-b64042d844cf service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=7877e643-fe9e-4041-91fa-3a4f44279fd2&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0734386 May 24 11:20:32.497582 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ceb423dd-e7a0-434b-839d-7c4ebd77bfc1 req-0c97d71f-a22b-4929-84c6-e81dcf6cee67 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1277370 May 24 11:20:32.648161 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ceb423dd-e7a0-434b-839d-7c4ebd77bfc1 req-209b6dc6-9b86-4dba-8125-8e9175c23655 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1409419 May 24 11:20:32.769674 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b7f390c6-bc37-4986-b949-34e52a26b61b req-de29b5f6-b4be-4eae-af1e-d72164d220a0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:32.770125 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58068) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:32.770594 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-b7f390c6-bc37-4986-b949-34e52a26b61b req-de29b5f6-b4be-4eae-af1e-d72164d220a0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Synchronizing usage tracker for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:20:32.787729 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-b7f390c6-bc37-4986-b949-34e52a26b61b req-de29b5f6-b4be-4eae-af1e-d72164d220a0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Unset dirty status for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:20:32.788917 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b7f390c6-bc37-4986-b949-34e52a26b61b req-de29b5f6-b4be-4eae-af1e-d72164d220a0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.1095600 May 24 11:20:32.836464 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b7f390c6-bc37-4986-b949-34e52a26b61b req-2f45d98e-c8ac-4272-aea8-4983323af10a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0417576 May 24 11:20:32.895053 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2a8b2894-adf8-4e5d-bfc9-a095d2f830c8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:32.911281 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2a8b2894-adf8-4e5d-bfc9-a095d2f830c8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:32.915967 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2a8b2894-adf8-4e5d-bfc9-a095d2f830c8 admin admin] Usage tracker for resource:security_group_rule and tenant:a29f94774e3f49eca30854e6be6bf8ed is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:32.919269 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2a8b2894-adf8-4e5d-bfc9-a095d2f830c8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:32.928971 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2a8b2894-adf8-4e5d-bfc9-a095d2f830c8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:33.020255 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2a8b2894-adf8-4e5d-bfc9-a095d2f830c8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:33.024140 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2a8b2894-adf8-4e5d-bfc9-a095d2f830c8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:33.031018 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3c04653b-3f7e-4db1-8703-df6c4b9145ae admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:33.071868 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-2a8b2894-adf8-4e5d-bfc9-a095d2f830c8 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=a29f94774e3f49eca30854e6be6bf8ed&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3005354 May 24 11:20:33.072863 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2a8b2894-adf8-4e5d-bfc9-a095d2f830c8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ee157e61-b44b-4a9c-a7a3-3d1241895c9d,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:33.075465 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 58076) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:33.076809 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2a8b2894-adf8-4e5d-bfc9-a095d2f830c8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:33.137969 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-56891f63-e04e-46ce-baa9-f3ce23f33809 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:33.163960 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-56891f63-e04e-46ce-baa9-f3ce23f33809 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:33.220698 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-56891f63-e04e-46ce-baa9-f3ce23f33809 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:33.224468 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-03ce19b6-053c-47fe-8f4c-2c24a5d7c8bc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:33.237057 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-56891f63-e04e-46ce-baa9-f3ce23f33809 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ee157e61-b44b-4a9c-a7a3-3d1241895c9d,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:33.239977 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-56891f63-e04e-46ce-baa9-f3ce23f33809 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:33.242257 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-56891f63-e04e-46ce-baa9-f3ce23f33809 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:33.243021 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-56891f63-e04e-46ce-baa9-f3ce23f33809 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/ee157e61-b44b-4a9c-a7a3-3d1241895c9d HTTP/1.1" status: 204 len: 168 time: 0.1668909 May 24 11:20:33.584310 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 9376988c-0749-43f3-a506-c8c29a15cbcf details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:20:33.721066 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=9376988c-0749-43f3-a506-c8c29a15cbcf,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:20:33.735168 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'port_id': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'mac_address': 'fa:16:3e:aa:5b:b7', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 500, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:20:33.796709 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:33.815177 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 9376988c-0749-43f3-a506-c8c29a15cbcf {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:20:33.831186 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=9376988c-0749-43f3-a506-c8c29a15cbcf,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:20:33.835601 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:33.837574 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-d71c3a32-d104-46c1-8f8a-0c765c2e8c4a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:33.908142 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=9376988c-0749-43f3-a506-c8c29a15cbcf,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:33.912396 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:34.053451 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55de4996-3fe5-458a-9947-01d99ea8c0aa req-f0857f3a-e1df-4330-8e04-e2c7dc43d89a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:34.057542 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-55de4996-3fe5-458a-9947-01d99ea8c0aa req-f0857f3a-e1df-4330-8e04-e2c7dc43d89a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 945 time: 0.0918121 May 24 11:20:34.105420 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['9376988c-0749-43f3-a506-c8c29a15cbcf'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:20:34.106023 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-55de4996-3fe5-458a-9947-01d99ea8c0aa req-ded7bc60-ccf1-430c-b4b1-2fbdefa90a71 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0451245 May 24 11:20:35.299328 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1aac672c-7c9c-4ad5-acd5-7a1e3b878fb1 req-f07be756-56b8-41dd-9e74-1a105ba3fc05 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:35.301236 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1aac672c-7c9c-4ad5-acd5-7a1e3b878fb1 req-f07be756-56b8-41dd-9e74-1a105ba3fc05 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 945 time: 0.0844469 May 24 11:20:35.347705 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1aac672c-7c9c-4ad5-acd5-7a1e3b878fb1 req-677b5334-bd51-4802-b236-f99dcd9e7158 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0398831 May 24 11:20:35.614637 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 9376988c-0749-43f3-a506-c8c29a15cbcf up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:20:35.683453 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 9376988c-0749-43f3-a506-c8c29a15cbcf completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:20:35.688321 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 9376988c-0749-43f3-a506-c8c29a15cbcf triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:20:35.688690 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:35.812767 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:35.842814 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:20:35.853310 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=9376988c-0749-43f3-a506-c8c29a15cbcf,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:20:35.860946 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:35.862744 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-46d24114-42cf-4dc3-ad05-93eb4a0ec306 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:35.901763 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'code': 200} May 24 11:20:35.940723 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=9376988c-0749-43f3-a506-c8c29a15cbcf,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:35.943534 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:36.154124 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f992246-3883-4aa2-a496-410392b5227b req-e97b6c32-d4e0-43be-abc3-817e9050aa1d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:36.155634 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2f992246-3883-4aa2-a496-410392b5227b req-e97b6c32-d4e0-43be-abc3-817e9050aa1d service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 1099 time: 0.0705917 May 24 11:20:36.211900 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2f992246-3883-4aa2-a496-410392b5227b req-aa443963-0aca-47cf-b81b-6c5454df2830 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0418870 May 24 11:20:36.536007 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3c5b39c9-3bca-4901-b0d0-2f53b02fa02e req-e2922bbb-7350-4b27-8cde-f4bae05c8872 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:36.537617 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3c5b39c9-3bca-4901-b0d0-2f53b02fa02e req-e2922bbb-7350-4b27-8cde-f4bae05c8872 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.0792661 May 24 11:20:36.585396 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3c5b39c9-3bca-4901-b0d0-2f53b02fa02e req-4218388e-4590-4707-b185-c24e4c940fde tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0413597 May 24 11:20:37.524575 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9f374c02-82b5-47b5-a0d4-46d1b95e5209 req-7aad2316-0708-45dd-82fa-9bb16c527808 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:37.526187 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9f374c02-82b5-47b5-a0d4-46d1b95e5209 req-7aad2316-0708-45dd-82fa-9bb16c527808 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.0892253 May 24 11:20:37.577368 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9f374c02-82b5-47b5-a0d4-46d1b95e5209 req-f3cc9e2a-a580-45b7-8c9d-01b1050187e6 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0433211 May 24 11:20:37.777244 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0ab05bf-0ddc-4132-9142-277bd8e56a76 req-2f971d96-6353-4e19-b0b0-f7113f12e0f8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:37.778625 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b0ab05bf-0ddc-4132-9142-277bd8e56a76 req-2f971d96-6353-4e19-b0b0-f7113f12e0f8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.0805392 May 24 11:20:37.826623 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b0ab05bf-0ddc-4132-9142-277bd8e56a76 req-4f8bfcf7-42ef-4e1d-b732-0f61dea6aa6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0424271 May 24 11:20:38.292679 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:20:38.762265 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f969c77-2678-49e9-89f8-a368a76a4dd8 req-822938db-a50a-4edb-9ca6-9a55c70030c6 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:38.763727 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1f969c77-2678-49e9-89f8-a368a76a4dd8 req-822938db-a50a-4edb-9ca6-9a55c70030c6 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.0789478 May 24 11:20:38.810969 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1f969c77-2678-49e9-89f8-a368a76a4dd8 req-fa7b3a45-dcfa-4e10-852d-6e70547839d4 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0406601 May 24 11:20:39.023375 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5563dcaf-7455-41df-8713-96155058df1b req-1a3f9985-2e9f-41ea-a19f-ce5019de2603 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:39.024846 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5563dcaf-7455-41df-8713-96155058df1b req-1a3f9985-2e9f-41ea-a19f-ce5019de2603 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.0864651 May 24 11:20:39.070076 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5563dcaf-7455-41df-8713-96155058df1b req-973d3f49-7699-45b8-a605-f32dbb21c452 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0395637 May 24 11:20:39.908817 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1090b558-7f8d-4e56-ae3f-726faabc67cf req-ec6a43da-329a-4821-b0c7-36f5fc146694 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:39.910898 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1090b558-7f8d-4e56-ae3f-726faabc67cf req-ec6a43da-329a-4821-b0c7-36f5fc146694 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=788050c389224d17b163f24bb67902bf&device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 1137 time: 0.0752573 May 24 11:20:40.191295 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3be81acc-7d0d-44fa-985e-048c4213a0d3 req-7af5b420-a37d-4067-acc2-ed9e907daf6f tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:40.192912 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3be81acc-7d0d-44fa-985e-048c4213a0d3 req-7af5b420-a37d-4067-acc2-ed9e907daf6f tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.2757607 May 24 11:20:40.257059 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1090b558-7f8d-4e56-ae3f-726faabc67cf req-e61a3bdb-9c00-47ab-a961-3ea303f1bfe8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:40.258602 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1090b558-7f8d-4e56-ae3f-726faabc67cf req-e61a3bdb-9c00-47ab-a961-3ea303f1bfe8 service neutron] 10.222.0.93 "GET /v2.0/networks?id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 HTTP/1.1" status: 200 len: 915 time: 0.3400121 May 24 11:20:40.331361 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3be81acc-7d0d-44fa-985e-048c4213a0d3 req-51bd0cc3-38b3-495d-8760-69b698eaf8c1 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.1208804 May 24 11:20:40.396236 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1090b558-7f8d-4e56-ae3f-726faabc67cf req-6bffdec1-6a39-4219-a9f4-399f87b93bdc service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010 HTTP/1.1" status: 200 len: 1001 time: 0.1283576 May 24 11:20:40.404086 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b7366998-cc55-43f4-8039-8b92a671cec5 req-6078f8c7-9d1c-4041-8834-b6ff319758ef tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:40.405689 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b7366998-cc55-43f4-8039-8b92a671cec5 req-6078f8c7-9d1c-4041-8834-b6ff319758ef tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1898890 May 24 11:20:40.460452 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b7366998-cc55-43f4-8039-8b92a671cec5 req-a449f8ca-ff1b-4ecc-810a-96e432036f57 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0481751 May 24 11:20:40.471196 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1090b558-7f8d-4e56-ae3f-726faabc67cf req-54856141-816d-47e9-87ae-fbd9e3a3deaa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:40.473326 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1090b558-7f8d-4e56-ae3f-726faabc67cf req-54856141-816d-47e9-87ae-fbd9e3a3deaa service neutron] 10.222.0.93 "GET /v2.0/subnets?id=6a80c579-20d5-46c7-8c07-6438f1367afd HTTP/1.1" status: 200 len: 853 time: 0.0690632 May 24 11:20:40.563793 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1090b558-7f8d-4e56-ae3f-726faabc67cf req-5b919550-5bb3-4bc7-8659-cf301d020dc9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:40.565549 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1090b558-7f8d-4e56-ae3f-726faabc67cf req-5b919550-5bb3-4bc7-8659-cf301d020dc9 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0817475 May 24 11:20:40.720467 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1090b558-7f8d-4e56-ae3f-726faabc67cf req-ed373d53-5091-4948-8570-527a0fb985d2 service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1450040 May 24 11:20:40.875652 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1090b558-7f8d-4e56-ae3f-726faabc67cf req-6c11ed5c-0b84-471a-8589-130fac907f0d service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1472163 May 24 11:20:41.567089 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2a1b4e46-654c-4b13-a2bf-901c828541a3 req-88234058-132b-4768-a0b7-8d22d3adf98c tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:41.568633 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2a1b4e46-654c-4b13-a2bf-901c828541a3 req-88234058-132b-4768-a0b7-8d22d3adf98c tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.1129887 May 24 11:20:41.623726 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2a1b4e46-654c-4b13-a2bf-901c828541a3 req-590b4417-c75f-4e76-a29e-57d9f9094c8b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0479686 May 24 11:20:41.661872 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dcf7a197-5dac-4f19-b144-f42dfcfc16b3 req-49c4a944-01e5-4fb7-b165-bcb17e0f824a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:41.664028 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dcf7a197-5dac-4f19-b144-f42dfcfc16b3 req-49c4a944-01e5-4fb7-b165-bcb17e0f824a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.0867383 May 24 11:20:41.712895 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dcf7a197-5dac-4f19-b144-f42dfcfc16b3 req-eef53d79-4382-4c3d-a927-5d6fef532c7f tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0434978 May 24 11:20:41.928237 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e0f1c4b-cc92-4a82-b622-9b354b4c674c req-3af75169-c62a-4570-8804-b5dcaa574495 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:41.929795 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6e0f1c4b-cc92-4a82-b622-9b354b4c674c req-3af75169-c62a-4570-8804-b5dcaa574495 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1061099 May 24 11:20:41.940070 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58214) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:41.945996 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-abb9acce-8432-45b1-bb6f-e331a73a010e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Request body: {'floatingip': {'port_id': '9376988c-0749-43f3-a506-c8c29a15cbcf'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:20:42.038008 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-abb9acce-8432-45b1-bb6f-e331a73a010e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:42.717619 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-abb9acce-8432-45b1-bb6f-e331a73a010e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:42.723694 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-abb9acce-8432-45b1-bb6f-e331a73a010e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:42.968078 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [None req-abb9acce-8432-45b1-bb6f-e331a73a010e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Floating IP 2f97fb4c-710a-4b28-a681-dfc09baedda8 associated. External IP: 172.27.0.70, port: 9376988c-0749-43f3-a506-c8c29a15cbcf. May 24 11:20:43.087955 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-11c4f246-f610-4a4f-a6ba-a9f8a5449179 req-073d409d-ed40-4538-9c06-7780c9726c1e tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:43.089523 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-11c4f246-f610-4a4f-a6ba-a9f8a5449179 req-073d409d-ed40-4538-9c06-7780c9726c1e tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.3039382 May 24 11:20:43.175282 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-11c4f246-f610-4a4f-a6ba-a9f8a5449179 req-fbc34274-9537-44cc-ad2b-fcd4eb73ab5b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0760770 May 24 11:20:43.533231 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-abb9acce-8432-45b1-bb6f-e331a73a010e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Router c2ffed60-7ad5-4215-9e01-680707fb98d6 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:20:43.791047 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-abb9acce-8432-45b1-bb6f-e331a73a010e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:20:43.805003 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d1cc18bd-3810-4c4f-acf4-dce0c3edc916 None None] Sync routers for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:20:43.813914 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-abb9acce-8432-45b1-bb6f-e331a73a010e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:43.852046 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d1cc18bd-3810-4c4f-acf4-dce0c3edc916 None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:20:43.852731 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-d1cc18bd-3810-4c4f-acf4-dce0c3edc916 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:20:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:43.877342 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-abb9acce-8432-45b1-bb6f-e331a73a010e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "PUT /v2.0/floatingips/2f97fb4c-710a-4b28-a681-dfc09baedda8 HTTP/1.1" status: 200 len: 991 time: 1.9360602 May 24 11:20:43.878189 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:20:43.906310 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58250) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:43.941115 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'status': 'completed', 'code': 200} May 24 11:20:44.057801 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0395ea42-b891-41c1-ad9f-a48b4776eb79 req-91829c28-2235-4198-8aa1-21f060b2a011 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:44.058545 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-44d69bb9-fb2d-44d8-acf2-4a6f561173a9 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:44.059398 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0395ea42-b891-41c1-ad9f-a48b4776eb79 req-91829c28-2235-4198-8aa1-21f060b2a011 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 1099 time: 0.0865045 May 24 11:20:44.060927 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-d1cc18bd-3810-4c4f-acf4-dce0c3edc916 None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:20:44.077639 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-44d69bb9-fb2d-44d8-acf2-4a6f561173a9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:44.081882 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-44d69bb9-fb2d-44d8-acf2-4a6f561173a9 admin admin] Usage tracker for resource:security_group_rule and tenant:bcc49c7c5c2b410fbe2546a3503ee706 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:44.084806 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-44d69bb9-fb2d-44d8-acf2-4a6f561173a9 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:44.091462 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-44d69bb9-fb2d-44d8-acf2-4a6f561173a9 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:44.186359 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-44d69bb9-fb2d-44d8-acf2-4a6f561173a9 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:44.190731 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-44d69bb9-fb2d-44d8-acf2-4a6f561173a9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:44.200508 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e69db118-0bd4-4e49-b3b3-07301ea771ef admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:44.200959 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0395ea42-b891-41c1-ad9f-a48b4776eb79 req-e6724843-d318-4b60-9c19-013fc13df93b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:44.202509 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0395ea42-b891-41c1-ad9f-a48b4776eb79 req-e6724843-d318-4b60-9c19-013fc13df93b service neutron] 10.222.0.93 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 917 time: 0.1357648 May 24 11:20:44.242863 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-44d69bb9-fb2d-44d8-acf2-4a6f561173a9 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=bcc49c7c5c2b410fbe2546a3503ee706&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3350382 May 24 11:20:44.246217 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58258) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:44.250880 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-44d69bb9-fb2d-44d8-acf2-4a6f561173a9 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=65b4335b-f5e4-4d24-bcac-ccf5971794d8,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:44.254666 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-44d69bb9-fb2d-44d8-acf2-4a6f561173a9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:44.336345 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0395ea42-b891-41c1-ad9f-a48b4776eb79 req-9e74974e-599b-4bdc-9709-7cb28e7ce321 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=9376988c-0749-43f3-a506-c8c29a15cbcf HTTP/1.1" status: 200 len: 999 time: 0.1237917 May 24 11:20:44.379603 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-65de9ad5-0956-4d6f-b1a0-a9a40964f565 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:44.413861 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-65de9ad5-0956-4d6f-b1a0-a9a40964f565 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:44.414720 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0395ea42-b891-41c1-ad9f-a48b4776eb79 req-510f3dab-14f3-459e-9e6b-b76d70582354 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:44.416214 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0395ea42-b891-41c1-ad9f-a48b4776eb79 req-510f3dab-14f3-459e-9e6b-b76d70582354 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=12ff35e8-9af8-4339-95c9-c62c90d8c6fd HTTP/1.1" status: 200 len: 855 time: 0.0732636 May 24 11:20:44.460541 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-65b2cbc2-b9ab-4d0b-821d-74e3f13d3d06 req-fe8c181f-6c98-4797-9b10-57ef958b1411 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:44.462170 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-65b2cbc2-b9ab-4d0b-821d-74e3f13d3d06 req-fe8c181f-6c98-4797-9b10-57ef958b1411 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.1573689 May 24 11:20:44.465944 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-d1cc18bd-3810-4c4f-acf4-dce0c3edc916 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '2f97fb4c-710a-4b28-a681-dfc09baedda8', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'floating_ip_address': '172.27.0.70', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'port_id': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 467, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'mac_address': 'fa:16:3e:aa:5b:b7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:20:25Z', 'updated_at': '2021-05-24T11:20:42Z', 'revision_number': 1, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:26Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:44.473804 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d1cc18bd-3810-4c4f-acf4-dce0c3edc916 None None] Checking router: c2ffed60-7ad5-4215-9e01-680707fb98d6 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:20:44.488573 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d1cc18bd-3810-4c4f-acf4-dce0c3edc916 None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:20:44.489876 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-d1cc18bd-3810-4c4f-acf4-dce0c3edc916 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:20:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:44.501973 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-65de9ad5-0956-4d6f-b1a0-a9a40964f565 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:44.509412 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-33de704c-98a7-4007-8ab8-674aa81a055e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:44.511760 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0395ea42-b891-41c1-ad9f-a48b4776eb79 req-9bcbe59f-b6cb-41d9-97f1-ed6854e70c42 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:44.521043 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0395ea42-b891-41c1-ad9f-a48b4776eb79 req-9bcbe59f-b6cb-41d9-97f1-ed6854e70c42 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=7877e643-fe9e-4041-91fa-3a4f44279fd2&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0987122 May 24 11:20:44.532549 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-65de9ad5-0956-4d6f-b1a0-a9a40964f565 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=65b4335b-f5e4-4d24-bcac-ccf5971794d8,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:44.559987 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-65de9ad5-0956-4d6f-b1a0-a9a40964f565 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:44.563761 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-65de9ad5-0956-4d6f-b1a0-a9a40964f565 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:44.565042 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-65de9ad5-0956-4d6f-b1a0-a9a40964f565 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/65b4335b-f5e4-4d24-bcac-ccf5971794d8 HTTP/1.1" status: 204 len: 168 time: 0.3178041 May 24 11:20:44.573968 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-65b2cbc2-b9ab-4d0b-821d-74e3f13d3d06 req-43eb59cb-3d84-410c-9a30-853e91e73a94 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0898774 May 24 11:20:44.695192 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0395ea42-b891-41c1-ad9f-a48b4776eb79 req-30b2ca73-39c2-4c82-8414-53b14f75c3b3 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1656473 May 24 11:20:44.707724 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-d1cc18bd-3810-4c4f-acf4-dce0c3edc916 None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:20:44.854129 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0395ea42-b891-41c1-ad9f-a48b4776eb79 req-d7ed1668-a28e-4f51-aa9b-d8296147ee39 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1453593 May 24 11:20:45.113500 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-d1cc18bd-3810-4c4f-acf4-dce0c3edc916 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '2f97fb4c-710a-4b28-a681-dfc09baedda8', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'floating_ip_address': '172.27.0.70', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'port_id': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 467, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'mac_address': 'fa:16:3e:aa:5b:b7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:20:25Z', 'updated_at': '2021-05-24T11:20:42Z', 'revision_number': 1, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:26Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:20:45.119496 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d1cc18bd-3810-4c4f-acf4-dce0c3edc916 None None] The sync data for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi is: [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '2f97fb4c-710a-4b28-a681-dfc09baedda8', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'floating_ip_address': '172.27.0.70', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'port_id': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 467, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'mac_address': 'fa:16:3e:aa:5b:b7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:20:25Z', 'updated_at': '2021-05-24T11:20:42Z', 'revision_number': 1, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:26Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:20:45.347646 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8f2995bd-59b0-420a-88b8-ad2aa812c161 None None] New status for floating IP 2f97fb4c-710a-4b28-a681-dfc09baedda8: ACTIVE {{(pid=91822) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:20:45.788824 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a8d5bb08-db60-43f5-910c-18ee03e7301d req-40b165c2-a00d-4ba2-8dfc-ceb019116445 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:45.790451 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a8d5bb08-db60-43f5-910c-18ee03e7301d req-40b165c2-a00d-4ba2-8dfc-ceb019116445 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.0951865 May 24 11:20:45.838601 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a8d5bb08-db60-43f5-910c-18ee03e7301d req-1fd83953-9eff-4f54-89aa-0111e5bdb93b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0422564 May 24 11:20:47.050247 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-31aad77b-c6c6-4d34-8615-35344806dad6 req-83396776-bde8-4f6c-99e2-3cee2f109b5c tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:47.052500 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-31aad77b-c6c6-4d34-8615-35344806dad6 req-83396776-bde8-4f6c-99e2-3cee2f109b5c tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.0922780 May 24 11:20:47.105886 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-31aad77b-c6c6-4d34-8615-35344806dad6 req-3038bbcb-de11-4c46-a4bb-9f5fc6b2f9cc tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0483174 May 24 11:20:48.305641 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4dd8480b-6e28-4be7-87fa-f5f6931e20d5 req-9a075fe4-11a8-475f-97a8-56dfccec3c59 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:48.307298 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4dd8480b-6e28-4be7-87fa-f5f6931e20d5 req-9a075fe4-11a8-475f-97a8-56dfccec3c59 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.0862222 May 24 11:20:48.361111 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4dd8480b-6e28-4be7-87fa-f5f6931e20d5 req-33b08d2b-26d2-4141-ad38-d24557922f5e tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0455966 May 24 11:20:49.611806 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ea6c4e8b-73d4-450a-a8fc-47b9e11eb54b req-d2496277-4c58-4266-82e3-70c82b14f4f0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:49.613570 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ea6c4e8b-73d4-450a-a8fc-47b9e11eb54b req-d2496277-4c58-4266-82e3-70c82b14f4f0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.1135492 May 24 11:20:49.668873 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ea6c4e8b-73d4-450a-a8fc-47b9e11eb54b req-2565654b-a58f-4e02-95ae-bef5e69b88d0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0491693 May 24 11:20:50.893899 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f121679b-2c35-417f-91fb-4d241f64ad5a req-d44c6b37-1ec7-4300-8b96-97c0a26e1669 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:50.896014 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f121679b-2c35-417f-91fb-4d241f64ad5a req-d44c6b37-1ec7-4300-8b96-97c0a26e1669 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.1090257 May 24 11:20:50.949520 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f121679b-2c35-417f-91fb-4d241f64ad5a req-2c08bc8b-f4f4-4750-aeba-7678817cd990 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0436692 May 24 11:20:52.010659 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:52.162455 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67f6b19c-c353-4c8e-babd-648d9c7479ce req-fa867310-5d68-4c32-ae1e-142a027d89d3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:52.163985 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-67f6b19c-c353-4c8e-babd-648d9c7479ce req-fa867310-5d68-4c32-ae1e-142a027d89d3 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.1008325 May 24 11:20:52.225991 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-67f6b19c-c353-4c8e-babd-648d9c7479ce req-c74d25ad-3fcd-423c-843d-736a3a23b569 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0512238 May 24 11:20:53.415395 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-63c6b16d-de1e-4ec2-b27d-36266ec6a3b9 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:53.464446 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-03c02ad8-6aaf-4e57-8c0a-308596940604 req-a115239f-1f12-4b5e-9a5b-d68c67e3235b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:53.465930 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-03c02ad8-6aaf-4e57-8c0a-308596940604 req-a115239f-1f12-4b5e-9a5b-d68c67e3235b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.1225019 May 24 11:20:53.558797 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-03c02ad8-6aaf-4e57-8c0a-308596940604 req-a57b635b-0d80-4248-928c-29688d14c1f6 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0858924 May 24 11:20:54.769102 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-69c8f928-5144-4e0f-86fd-890155da355a req-f819da78-3ac3-4c33-b863-d139f2ae130b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:20:54.770934 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-69c8f928-5144-4e0f-86fd-890155da355a req-f819da78-3ac3-4c33-b863-d139f2ae130b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.0866382 May 24 11:20:54.827660 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-69c8f928-5144-4e0f-86fd-890155da355a req-b9b62696-1034-49b4-9fbe-a8df705e177e tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&id=9286a221-16dd-4047-bf8c-6373837405d0&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0470397 May 24 11:20:55.002396 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58454) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:55.141224 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-6bcd0690-0d52-43ed-97f9-f2c0f7dc1885 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:55.162518 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-6bcd0690-0d52-43ed-97f9-f2c0f7dc1885 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:55.166345 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-6bcd0690-0d52-43ed-97f9-f2c0f7dc1885 admin admin] Usage tracker for resource:security_group_rule and tenant:0e01ab8151944a34b022481db9b55ff7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:55.169136 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-6bcd0690-0d52-43ed-97f9-f2c0f7dc1885 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:55.175305 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-6bcd0690-0d52-43ed-97f9-f2c0f7dc1885 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:55.262001 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-6bcd0690-0d52-43ed-97f9-f2c0f7dc1885 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:55.266002 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-6bcd0690-0d52-43ed-97f9-f2c0f7dc1885 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:55.273889 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-be2d3c36-2728-4b66-a96d-6997a9660bf2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:55.316073 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6bcd0690-0d52-43ed-97f9-f2c0f7dc1885 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e32d9a5e-98b9-4b48-a567-e145e96dbedb,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:55.319313 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-6bcd0690-0d52-43ed-97f9-f2c0f7dc1885 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:55.321934 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-6bcd0690-0d52-43ed-97f9-f2c0f7dc1885 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=0e01ab8151944a34b022481db9b55ff7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3185687 May 24 11:20:55.324914 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58462) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:55.478231 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a9536fa8-185d-47eb-b959-f1de451f3aef admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:55.504760 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a9536fa8-185d-47eb-b959-f1de451f3aef admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:55.559738 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a9536fa8-185d-47eb-b959-f1de451f3aef admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:55.565570 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-44d69bb9-fb2d-44d8-acf2-4a6f561173a9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:55.579854 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a9536fa8-185d-47eb-b959-f1de451f3aef admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e32d9a5e-98b9-4b48-a567-e145e96dbedb,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:55.582355 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a9536fa8-185d-47eb-b959-f1de451f3aef admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:55.583316 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a9536fa8-185d-47eb-b959-f1de451f3aef admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:55.585512 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-a9536fa8-185d-47eb-b959-f1de451f3aef admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/e32d9a5e-98b9-4b48-a567-e145e96dbedb HTTP/1.1" status: 204 len: 168 time: 0.2585089 May 24 11:20:55.761384 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:55.868647 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-41e30f47-31ae-470e-8aa9-21075f64ecb5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:55.890139 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 58478) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:56.056230 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3a9b903c-8fff-4e59-9d3b-7e4519b33767 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:56.071134 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-3a9b903c-8fff-4e59-9d3b-7e4519b33767 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:56.075192 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-3a9b903c-8fff-4e59-9d3b-7e4519b33767 admin admin] Usage tracker for resource:security_group_rule and tenant:b3be65b523c744b18d38c401b998645d is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:56.078092 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-3a9b903c-8fff-4e59-9d3b-7e4519b33767 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:56.083773 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-3a9b903c-8fff-4e59-9d3b-7e4519b33767 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:56.169418 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3a9b903c-8fff-4e59-9d3b-7e4519b33767 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:56.174072 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3a9b903c-8fff-4e59-9d3b-7e4519b33767 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:56.273288 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-a28c1c64-c5d1-4053-b6eb-4a070605d220 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:56.315766 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-3a9b903c-8fff-4e59-9d3b-7e4519b33767 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=b3be65b523c744b18d38c401b998645d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4242203 May 24 11:20:56.316697 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3a9b903c-8fff-4e59-9d3b-7e4519b33767 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e4b8179c-a0fd-45d7-ab1f-5a7035996e28,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:56.318646 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58486) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:56.319966 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-3a9b903c-8fff-4e59-9d3b-7e4519b33767 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:56.382797 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-916b852d-34f1-4b4e-bb52-229a63e72679 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:56.408393 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-916b852d-34f1-4b4e-bb52-229a63e72679 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:56.462805 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-916b852d-34f1-4b4e-bb52-229a63e72679 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:56.466384 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-44341d82-6356-459c-9062-c7dbff0a6408 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:56.477922 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-916b852d-34f1-4b4e-bb52-229a63e72679 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e4b8179c-a0fd-45d7-ab1f-5a7035996e28,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:56.482320 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-916b852d-34f1-4b4e-bb52-229a63e72679 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:56.484237 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-916b852d-34f1-4b4e-bb52-229a63e72679 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:56.485920 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-916b852d-34f1-4b4e-bb52-229a63e72679 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/e4b8179c-a0fd-45d7-ab1f-5a7035996e28 HTTP/1.1" status: 204 len: 168 time: 0.1661713 May 24 11:20:56.779689 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58494) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:56.799095 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c533b884-2733-4003-a0c6-f02d300b6eac admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:56.814323 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-c533b884-2733-4003-a0c6-f02d300b6eac admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:56.818385 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-c533b884-2733-4003-a0c6-f02d300b6eac admin admin] Usage tracker for resource:security_group_rule and tenant:b9506d9cca4645c085357b3b7f6b9800 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:56.821339 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-c533b884-2733-4003-a0c6-f02d300b6eac admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:56.827542 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-c533b884-2733-4003-a0c6-f02d300b6eac admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:56.928640 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c533b884-2733-4003-a0c6-f02d300b6eac admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:56.934262 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c533b884-2733-4003-a0c6-f02d300b6eac admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:56.947726 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-39ac601b-3640-471a-a88d-97114465f699 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:56.976822 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:57.013938 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-c533b884-2733-4003-a0c6-f02d300b6eac admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=b9506d9cca4645c085357b3b7f6b9800&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2326174 May 24 11:20:57.015290 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c533b884-2733-4003-a0c6-f02d300b6eac admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=107cff51-8478-467f-a0da-2b9da8effede,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:57.016555 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 58498) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:57.020420 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c533b884-2733-4003-a0c6-f02d300b6eac admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:57.084414 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-fa9bb883-0544-46e5-903c-40bb4cb0cfc1 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:57.114658 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-fa9bb883-0544-46e5-903c-40bb4cb0cfc1 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:57.188873 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-fa9bb883-0544-46e5-903c-40bb4cb0cfc1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:57.196856 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-add39e63-49b5-4d3e-aead-6a8297d6a735 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:57.216928 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fa9bb883-0544-46e5-903c-40bb4cb0cfc1 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=107cff51-8478-467f-a0da-2b9da8effede,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:57.227713 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-fa9bb883-0544-46e5-903c-40bb4cb0cfc1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:57.230098 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-fa9bb883-0544-46e5-903c-40bb4cb0cfc1 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:57.231805 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-fa9bb883-0544-46e5-903c-40bb4cb0cfc1 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/107cff51-8478-467f-a0da-2b9da8effede HTTP/1.1" status: 204 len: 168 time: 0.2139766 May 24 11:20:57.695051 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58508) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:57.725431 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0052ec49-c5c2-4727-b543-4f230606fb38 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:57.749189 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-0052ec49-c5c2-4727-b543-4f230606fb38 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:20:57.754096 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-0052ec49-c5c2-4727-b543-4f230606fb38 admin admin] Usage tracker for resource:security_group_rule and tenant:d0b10b00d4de4b0cad7f96fd52c849bf is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:20:57.756920 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-0052ec49-c5c2-4727-b543-4f230606fb38 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:20:57.762875 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-0052ec49-c5c2-4727-b543-4f230606fb38 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:20:57.884666 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0052ec49-c5c2-4727-b543-4f230606fb38 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:57.893611 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0052ec49-c5c2-4727-b543-4f230606fb38 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:57.903386 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9e66e005-946d-4ce8-aa83-282a745ef751 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:57.948186 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-0052ec49-c5c2-4727-b543-4f230606fb38 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=d0b10b00d4de4b0cad7f96fd52c849bf&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2517292 May 24 11:20:57.949200 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0052ec49-c5c2-4727-b543-4f230606fb38 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5e4dec61-5103-4174-bdd7-d0e8f179f5bb,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:57.952419 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-0052ec49-c5c2-4727-b543-4f230606fb38 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:57.956166 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58512) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:20:58.018341 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f815f17d-565e-47a3-9b43-685267af9168 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:58.045438 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f815f17d-565e-47a3-9b43-685267af9168 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:58.108554 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f815f17d-565e-47a3-9b43-685267af9168 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:58.112959 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e61a22c0-cbb5-46f3-9e2f-d19bd9d2f59a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:20:58.127298 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f815f17d-565e-47a3-9b43-685267af9168 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5e4dec61-5103-4174-bdd7-d0e8f179f5bb,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:20:58.129897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f815f17d-565e-47a3-9b43-685267af9168 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:20:58.143165 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f815f17d-565e-47a3-9b43-685267af9168 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:20:58.144790 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-f815f17d-565e-47a3-9b43-685267af9168 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/5e4dec61-5103-4174-bdd7-d0e8f179f5bb HTTP/1.1" status: 204 len: 168 time: 0.1874146 May 24 11:20:58.921042 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:02.356024 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58570) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:02.492703 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bbf79305-86f3-4218-bf0a-38a7a20e5faa admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:02.519788 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-bbf79305-86f3-4218-bf0a-38a7a20e5faa admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:21:02.525592 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-bbf79305-86f3-4218-bf0a-38a7a20e5faa admin admin] Usage tracker for resource:security_group_rule and tenant:573ce177142b433b8c581e9d4de87bcd is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:21:02.529616 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-bbf79305-86f3-4218-bf0a-38a7a20e5faa admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:21:02.537428 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-bbf79305-86f3-4218-bf0a-38a7a20e5faa admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:21:02.630276 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bbf79305-86f3-4218-bf0a-38a7a20e5faa admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:02.634075 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bbf79305-86f3-4218-bf0a-38a7a20e5faa admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:02.641933 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2a8b2894-adf8-4e5d-bfc9-a095d2f830c8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:02.681320 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-bbf79305-86f3-4218-bf0a-38a7a20e5faa admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=573ce177142b433b8c581e9d4de87bcd&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3235605 May 24 11:21:02.682931 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bbf79305-86f3-4218-bf0a-38a7a20e5faa admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8ea0f18a-a512-453c-96e2-948719a9de0f,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:02.684091 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 58576) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:02.685937 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-bbf79305-86f3-4218-bf0a-38a7a20e5faa admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:02.766605 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e8d1acb4-9a4a-460e-877d-f32f4074f12d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:02.810139 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e8d1acb4-9a4a-460e-877d-f32f4074f12d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:02.870846 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e8d1acb4-9a4a-460e-877d-f32f4074f12d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:02.878383 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-5d9cf2b9-5770-44b9-a624-c0c67ca90e23 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:02.893937 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e8d1acb4-9a4a-460e-877d-f32f4074f12d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8ea0f18a-a512-453c-96e2-948719a9de0f,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:02.896943 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e8d1acb4-9a4a-460e-877d-f32f4074f12d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:02.899959 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e8d1acb4-9a4a-460e-877d-f32f4074f12d admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:02.906292 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-e8d1acb4-9a4a-460e-877d-f32f4074f12d admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/8ea0f18a-a512-453c-96e2-948719a9de0f HTTP/1.1" status: 204 len: 168 time: 0.2147982 May 24 11:21:07.276301 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5c693a7c-bf3a-4d68-9eb4-f3849921f145 req-cfe08ad3-3eaa-4a1e-877c-1ca283837e8e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:07.278560 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5c693a7c-bf3a-4d68-9eb4-f3849921f145 req-cfe08ad3-3eaa-4a1e-877c-1ca283837e8e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.3162117 May 24 11:21:07.351209 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5c693a7c-bf3a-4d68-9eb4-f3849921f145 req-e952ab0f-8700-4584-9c3b-c55080f8906b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0666275 May 24 11:21:08.550014 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f9595aa-36c5-481e-8b53-28f2d042a916 req-7376af85-1ce3-4044-82e9-29d0bcc7b2e8 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:08.551905 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1f9595aa-36c5-481e-8b53-28f2d042a916 req-7376af85-1ce3-4044-82e9-29d0bcc7b2e8 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.0849433 May 24 11:21:08.601941 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1f9595aa-36c5-481e-8b53-28f2d042a916 req-13444942-568e-4fdd-8376-15d3aefd8c0d tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0431738 May 24 11:21:09.396913 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-aea91ff4-19d5-4d80-8956-62e2475ca736 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:09.398597 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-aea91ff4-19d5-4d80-8956-62e2475ca736 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.93 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.1175239 May 24 11:21:09.569231 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Deleting port eab44150-44f7-4194-9b76-436d421794e8 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:21:09.569933 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:09.744663 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] For port eab44150-44f7-4194-9b76-436d421794e8, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=eab44150-44f7-4194-9b76-436d421794e8,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:09.745028 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:10.156109 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:10.156522 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:10.156836 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.l3_db [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Floating IP 39b51eeb-5978-4a5b-8a19-eaa131930a0b disassociated. External IP: 172.27.0.86, port: eab44150-44f7-4194-9b76-436d421794e8. May 24 11:21:10.157884 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Calling delete_port for eab44150-44f7-4194-9b76-436d421794e8 owned by compute:nova {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:21:10.171698 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d1869de-52c4-42e9-acfa-23241348716c req-ed45c9b3-17d2-4340-87e5-9a605fc7f436 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:10.173492 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1d1869de-52c4-42e9-acfa-23241348716c req-ed45c9b3-17d2-4340-87e5-9a605fc7f436 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 946 time: 0.2337708 May 24 11:21:10.241381 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1d1869de-52c4-42e9-acfa-23241348716c req-d919e2d0-c621-49af-8373-b7ddabccb50d tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=20a7f42b-2874-40d1-94c8-66d210ae913e&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0536342 May 24 11:21:10.367386 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:10.374391 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:10.426001 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event deleted for resources: {'Port': ['ID=eab44150-44f7-4194-9b76-436d421794e8,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:10.439929 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:10.440307 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:10.440690 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:10.441044 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:10.441455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:10.441455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:10.441455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:10.441455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:10.441455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:10.441455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:10.441455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:10.441455 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:10.447673 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:10.673354 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Router f76568b3-17ef-487b-beeb-3f592a58e138 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:21:11.039676 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:21:11.053627 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-03d3f4fc-2956-4e26-b524-716789a68063 None None] Sync routers for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:21:11.079233 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-03d3f4fc-2956-4e26-b524-716789a68063 None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:21:11.080496 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-03d3f4fc-2956-4e26-b524-716789a68063 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:20:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:11.140490 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] No DVR routers for this DVR port eab44150-44f7-4194-9b76-436d421794e8 on host hv-cinder-78409 {{(pid=91820) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:21:11.143817 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:11.155801 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'name': 'network-vif-deleted', 'tag': 'eab44150-44f7-4194-9b76-436d421794e8'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:21:11.164084 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-eeb554e8-f03e-49de-a755-eb924b1bc724 req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.93 "DELETE /v2.0/ports/eab44150-44f7-4194-9b76-436d421794e8 HTTP/1.1" status: 204 len: 173 time: 1.7575924 May 24 11:21:11.244518 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '728281d0-9d5e-4f30-93cb-a5fb92c4bc25', 'name': 'network-vif-deleted', 'tag': 'eab44150-44f7-4194-9b76-436d421794e8', 'status': 'completed', 'code': 200} May 24 11:21:11.496218 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-03d3f4fc-2956-4e26-b524-716789a68063 None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:21:11.603064 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b0b8790-e001-4bf6-b4db-777c6d074087 req-5e402da0-69b4-40d4-a6db-175ec973a56e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=728281d0-9d5e-4f30-93cb-a5fb92c4bc25 HTTP/1.1" status: 200 len: 210 time: 0.1534467 May 24 11:21:11.778314 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device eab44150-44f7-4194-9b76-436d421794e8 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:21:12.003075 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: eab44150-44f7-4194-9b76-436d421794e8 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:21:12.003578 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device eab44150-44f7-4194-9b76-436d421794e8 not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:21:12.066262 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-03d3f4fc-2956-4e26-b524-716789a68063 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:17Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:12.072980 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-03d3f4fc-2956-4e26-b524-716789a68063 None None] Checking router: f76568b3-17ef-487b-beeb-3f592a58e138 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:21:12.086158 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-03d3f4fc-2956-4e26-b524-716789a68063 None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:21:12.086800 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-03d3f4fc-2956-4e26-b524-716789a68063 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:20:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:12.337977 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-03d3f4fc-2956-4e26-b524-716789a68063 None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:21:12.752729 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58744) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:12.797373 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-03d3f4fc-2956-4e26-b524-716789a68063 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:17Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:12.801519 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-03d3f4fc-2956-4e26-b524-716789a68063 None None] The sync data for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi is: [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:17Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:21:12.851477 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:12.889949 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:12.954354 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:12.959583 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-65de9ad5-0956-4d6f-b1a0-a9a40964f565 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:12.977248 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=20a7f42b-2874-40d1-94c8-66d210ae913e,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:12.989418 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:12.989780 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:12.990348 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:12.990698 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:12.991215 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:12.991567 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:12.991931 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:12.992263 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:12.993068 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:12.993502 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:12.993853 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:12.994198 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:12.998234 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:13.009144 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:13.013798 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "DELETE /v2.0/security-groups/20a7f42b-2874-40d1-94c8-66d210ae913e HTTP/1.1" status: 204 len: 168 time: 0.2589509 May 24 11:21:13.014004 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 58748) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:13.045780 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.api.v2.resource [None req-57e267c8-bb00-4141-9305-472dda0279c6 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] show failed (client error): The resource could not be found. May 24 11:21:13.047613 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-57e267c8-bb00-4141-9305-472dda0279c6 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups/20a7f42b-2874-40d1-94c8-66d210ae913e HTTP/1.1" status: 404 len: 347 time: 0.0320399 May 24 11:21:13.050646 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58752) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:13.119012 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d11c2096-1ebb-4fb8-8f9d-a14a772db353 None None] New status for floating IP 39b51eeb-5978-4a5b-8a19-eaa131930a0b: DOWN {{(pid=91822) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:21:13.349923 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-d3098214-9cbd-419f-8781-4f2c374be97e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Deleting port 777aecd4-bf5c-4efa-a7e0-b3f488ba709c {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:21:13.349923 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3098214-9cbd-419f-8781-4f2c374be97e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:13.594607 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3098214-9cbd-419f-8781-4f2c374be97e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:13.595153 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3098214-9cbd-419f-8781-4f2c374be97e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:13.742164 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-d3098214-9cbd-419f-8781-4f2c374be97e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Calling delete_port for 777aecd4-bf5c-4efa-a7e0-b3f488ba709c owned by network:floatingip {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:21:13.877432 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3098214-9cbd-419f-8781-4f2c374be97e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:13.881642 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:13.931048 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d3098214-9cbd-419f-8781-4f2c374be97e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event deleted for resources: {'Port': ['ID=777aecd4-bf5c-4efa-a7e0-b3f488ba709c,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:13.965508 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d3098214-9cbd-419f-8781-4f2c374be97e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:14.036400 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3098214-9cbd-419f-8781-4f2c374be97e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.055701 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3098214-9cbd-419f-8781-4f2c374be97e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.056832 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-d3098214-9cbd-419f-8781-4f2c374be97e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "DELETE /v2.0/floatingips/39b51eeb-5978-4a5b-8a19-eaa131930a0b HTTP/1.1" status: 204 len: 168 time: 1.0050533 May 24 11:21:14.343970 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58766) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:14.349395 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Request body: {'security_group': {'name': 'tempest-securitygroup--326538570', 'description': 'tempest-description--1683972445'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:21:14.355229 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:21:14.359931 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Usage tracker for resource:security_group and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:21:14.362935 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:21:14.368707 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:21:14.381037 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.401448 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:21:14.405893 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Usage tracker for resource:security_group_rule and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:21:14.409161 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:21:14.414908 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:21:14.490335 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.494398 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.498981 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5119a5c1-bfd5-463d-b6fe-560160e74cb4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:14.541961 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event updated for resources: {'SecurityGroup': ['ID=19f35add-ecda-474f-b1ad-b25cb242f065,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:14.549275 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:14.550594 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.551901 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1779 time: 0.2068861 May 24 11:21:14.555250 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 58770) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:14.561047 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-bfb876a4-bce9-4a38-b4d5-2bbd90f9d108 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Request body: {'security_group_rule': {'security_group_id': '19f35add-ecda-474f-b1ad-b25cb242f065', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:21:14.567335 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-bfb876a4-bce9-4a38-b4d5-2bbd90f9d108 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:21:14.571641 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-bfb876a4-bce9-4a38-b4d5-2bbd90f9d108 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Usage tracker for resource:security_group_rule and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:21:14.574912 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-bfb876a4-bce9-4a38-b4d5-2bbd90f9d108 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:21:14.588639 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-bfb876a4-bce9-4a38-b4d5-2bbd90f9d108 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:21:14.603874 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-bfb876a4-bce9-4a38-b4d5-2bbd90f9d108 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.667554 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-bfb876a4-bce9-4a38-b4d5-2bbd90f9d108 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.672218 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-bfb876a4-bce9-4a38-b4d5-2bbd90f9d108 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.674238 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:14.702456 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bfb876a4-bce9-4a38-b4d5-2bbd90f9d108 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=404bf236-e87f-4d48-897e-b7615c81169e,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:14.705294 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-bfb876a4-bce9-4a38-b4d5-2bbd90f9d108 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:14.709402 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-bfb876a4-bce9-4a38-b4d5-2bbd90f9d108 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.714219 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-bfb876a4-bce9-4a38-b4d5-2bbd90f9d108 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.1543519 May 24 11:21:14.714219 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 58778) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:14.718842 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-63da69ae-04e3-4b4b-bb2c-24913d4b32fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Request body: {'security_group_rule': {'security_group_id': '19f35add-ecda-474f-b1ad-b25cb242f065', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:21:14.724910 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-63da69ae-04e3-4b4b-bb2c-24913d4b32fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:21:14.728834 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-63da69ae-04e3-4b4b-bb2c-24913d4b32fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Usage tracker for resource:security_group_rule and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:21:14.731758 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-63da69ae-04e3-4b4b-bb2c-24913d4b32fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:21:14.738096 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-63da69ae-04e3-4b4b-bb2c-24913d4b32fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:21:14.756786 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-63da69ae-04e3-4b4b-bb2c-24913d4b32fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.825682 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-63da69ae-04e3-4b4b-bb2c-24913d4b32fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.831134 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-63da69ae-04e3-4b4b-bb2c-24913d4b32fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.838376 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1f4380fe-d023-400f-bde2-8fdb384a40fe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:14.858858 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-63da69ae-04e3-4b4b-bb2c-24913d4b32fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=6dc203dd-838f-4ec2-a42f-9ca70b021a92,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:14.861696 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-63da69ae-04e3-4b4b-bb2c-24913d4b32fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:14.874017 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-63da69ae-04e3-4b4b-bb2c-24913d4b32fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:14.876148 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-63da69ae-04e3-4b4b-bb2c-24913d4b32fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.1610980 May 24 11:21:14.879193 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 58786) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:14.884270 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Request body: {'floatingip': {'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:21:14.890145 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:21:14.894423 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Usage tracker for resource:floatingip and tenant:b303f83a84794e5ba6a1bab9c76b429c is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:21:14.897255 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:21:14.903174 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:21:14.915766 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:15.092614 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:15.245621 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Allocated IP 172.27.0.194 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/03fa7a36-c8c0-4619-ba8d-280b53b6cfd9) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:21:15.291165 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:21:15.394626 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:15.427880 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:15.431439 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-c2e5d578-03ee-4e88-95b6-1c3d030baaca service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:15.496127 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:21:15.498794 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:21:15.502606 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:21:15.532454 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event updated for resources: {'Port': ['ID=03fa7a36-c8c0-4619-ba8d-280b53b6cfd9,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:15.538530 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:15.594153 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:15.652497 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:15.859867 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:15.864681 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:15.868988 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:15.975209 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event updated for resources: {'Port': ['ID=03fa7a36-c8c0-4619-ba8d-280b53b6cfd9,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:15.980217 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:15.988906 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:16.018250 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:16.019605 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-b988acc6-b209-4815-9a1f-07816ed455ca tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 693 time: 1.1394277 May 24 11:21:16.202020 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-0a53749e-f836-45b3-8584-233dfdcce69f req-504196ed-d5c3-406d-ada0-64e783bd1415 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Synchronizing usage tracker for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:21:16.219747 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-0a53749e-f836-45b3-8584-233dfdcce69f req-504196ed-d5c3-406d-ada0-64e783bd1415 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Unset dirty status for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:21:16.221110 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-504196ed-d5c3-406d-ada0-64e783bd1415 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?name=tempest-securitygroup--326538570&fields=id&tenant_id=b303f83a84794e5ba6a1bab9c76b429c HTTP/1.1" status: 200 len: 267 time: 0.0607400 May 24 11:21:16.321034 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 58812) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:16.446904 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ff5871d6-0ebc-42e5-b289-24b848a112a8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:16.467113 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-ff5871d6-0ebc-42e5-b289-24b848a112a8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:21:16.471821 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-ff5871d6-0ebc-42e5-b289-24b848a112a8 admin admin] Usage tracker for resource:security_group_rule and tenant:a2b48ada05444038a0241a1d5d4a4f85 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:21:16.474866 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-ff5871d6-0ebc-42e5-b289-24b848a112a8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:21:16.479397 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a53749e-f836-45b3-8584-233dfdcce69f req-f044157d-96a9-4c58-af34-8bc60ab3b7a6 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:16.481849 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-f044157d-96a9-4c58-af34-8bc60ab3b7a6 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/networks?id=1d92641a-d8f9-4ed6-9abd-f464bb26811d HTTP/1.1" status: 200 len: 811 time: 0.2532983 May 24 11:21:16.482843 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-ff5871d6-0ebc-42e5-b289-24b848a112a8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:21:16.502746 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-5e2cb11c-9fbf-4f7f-8f0a-2aed35b63243 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/quotas/b303f83a84794e5ba6a1bab9c76b429c HTTP/1.1" status: 200 len: 375 time: 0.0106509 May 24 11:21:16.570623 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ff5871d6-0ebc-42e5-b289-24b848a112a8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:16.576159 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ff5871d6-0ebc-42e5-b289-24b848a112a8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:16.592141 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6a3ba289-b539-4e83-9aff-c067d69e4b4e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:16.600764 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-0a53749e-f836-45b3-8584-233dfdcce69f req-850a8617-258f-4173-9fda-32ba5de01205 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Synchronizing usage tracker for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:21:16.618535 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-0a53749e-f836-45b3-8584-233dfdcce69f req-850a8617-258f-4173-9fda-32ba5de01205 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Unset dirty status for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:21:16.620182 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-850a8617-258f-4173-9fda-32ba5de01205 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=b303f83a84794e5ba6a1bab9c76b429c&fields=id HTTP/1.1" status: 200 len: 348 time: 0.1120501 May 24 11:21:16.633741 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-ff5871d6-0ebc-42e5-b289-24b848a112a8 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=a2b48ada05444038a0241a1d5d4a4f85&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3114121 May 24 11:21:16.637419 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 58818) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:16.650693 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ff5871d6-0ebc-42e5-b289-24b848a112a8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=905d8f80-86dd-4062-890b-59365f3e8221,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:16.655371 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ff5871d6-0ebc-42e5-b289-24b848a112a8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:16.713963 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cbebcf4a-61b7-4d38-969c-103a634c289b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:16.748280 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cbebcf4a-61b7-4d38-969c-103a634c289b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:16.812508 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cbebcf4a-61b7-4d38-969c-103a634c289b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:16.817125 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e5e84ec5-3ef8-4b1e-8292-cbd4450907a8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:16.832538 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cbebcf4a-61b7-4d38-969c-103a634c289b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=905d8f80-86dd-4062-890b-59365f3e8221,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:16.836870 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-cbebcf4a-61b7-4d38-969c-103a634c289b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:16.846315 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-26db0f16-678c-4a79-a34d-06b562b95017 service neutron] 10.222.0.18 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2118070 May 24 11:21:16.852641 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cbebcf4a-61b7-4d38-969c-103a634c289b admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:16.857432 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-cbebcf4a-61b7-4d38-969c-103a634c289b admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/905d8f80-86dd-4062-890b-59365f3e8221 HTTP/1.1" status: 204 len: 168 time: 0.2189584 May 24 11:21:16.989003 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-59e002d6-9cdf-4f2a-b167-1693c48126c7 service neutron] 10.222.0.18 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1415739 May 24 11:21:17.308652 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-72004856-5e31-4c10-ab20-d6fe4f047a88 req-b9dc284f-6757-4cd7-a3dc-b7bbd75cc565 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Synchronizing usage tracker for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:21:17.321051 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-72004856-5e31-4c10-ab20-d6fe4f047a88 req-b9dc284f-6757-4cd7-a3dc-b7bbd75cc565 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Unset dirty status for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:21:17.322292 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-72004856-5e31-4c10-ab20-d6fe4f047a88 req-b9dc284f-6757-4cd7-a3dc-b7bbd75cc565 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 210 time: 0.1756580 May 24 11:21:18.511157 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-93013483-78e9-4bb8-b469-bf4d2bb3af85 req-b07e1552-72e0-48cb-84eb-9b0fc0f0d95e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 210 time: 0.0723166 May 24 11:21:18.655000 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a53749e-f836-45b3-8584-233dfdcce69f req-5249d0ea-4f3f-4afb-ae85-e8f63e525b06 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:18.656929 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-5249d0ea-4f3f-4afb-ae85-e8f63e525b06 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.93 "GET /v2.0/networks?id=1d92641a-d8f9-4ed6-9abd-f464bb26811d HTTP/1.1" status: 200 len: 811 time: 0.1761396 May 24 11:21:18.714506 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-abc07db2-5e1a-4be3-8a8b-8d364c048572 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.93 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=b303f83a84794e5ba6a1bab9c76b429c HTTP/1.1" status: 200 len: 379 time: 0.0479982 May 24 11:21:18.725481 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Request body: {'port': {'device_id': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'admin_state_up': True, 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'security_groups': ['19f35add-ecda-474f-b1ad-b25cb242f065']}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:21:18.891513 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:21:18.901929 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:21:18.913887 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Made reservation on behalf of b303f83a84794e5ba6a1bab9c76b429c for: {'port': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:21:18.919566 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:19.042039 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-023028ac-49b5-436e-bd3d-c55713232cde req-35249f58-7274-4b45-b5f1-303cd373edf8 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1047187 May 24 11:21:19.118943 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Allocated IP 172.20.0.5 (1d92641a-d8f9-4ed6-9abd-f464bb26811d/2fc4be2e-875c-4d10-b857-a48f90c53cf2/6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:21:19.218594 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-023028ac-49b5-436e-bd3d-c55713232cde req-9ef409ce-44fd-438a-a9cc-7b7b9a40fb1a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:19.220223 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-023028ac-49b5-436e-bd3d-c55713232cde req-9ef409ce-44fd-438a-a9cc-7b7b9a40fb1a service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=788050c389224d17b163f24bb67902bf&device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 1137 time: 0.0845041 May 24 11:21:19.318047 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:19.337883 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Transition to ACTIVE for port object 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 will not be triggered until provisioned by entity DHCP. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:21:19.364436 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:19.370267 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:19.399049 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-023028ac-49b5-436e-bd3d-c55713232cde req-b10672c5-14f2-443b-bdd0-f301e77570b5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:19.400825 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-023028ac-49b5-436e-bd3d-c55713232cde req-b10672c5-14f2-443b-bdd0-f301e77570b5 service neutron] 10.222.0.93 "GET /v2.0/networks?id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 HTTP/1.1" status: 200 len: 915 time: 0.1523535 May 24 11:21:19.442409 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Network 1d92641a-d8f9-4ed6-9abd-f464bb26811d is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:21:19.445887 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:21:19.450198 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:21:19.480371 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event updated for resources: {'Port': ['ID=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:19.488475 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:19.519792 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:19.520520 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-023028ac-49b5-436e-bd3d-c55713232cde req-f05654aa-d448-4280-932e-ea0239f8a37b service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010 HTTP/1.1" status: 200 len: 1001 time: 0.1120932 May 24 11:21:19.520623 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:19.574740 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 0.8530991 May 24 11:21:19.589918 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Request body: {'port': {'device_id': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:21:19.593150 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-023028ac-49b5-436e-bd3d-c55713232cde req-731d4b03-bc3d-4bed-bc05-490a099b7477 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:19.595574 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-023028ac-49b5-436e-bd3d-c55713232cde req-731d4b03-bc3d-4bed-bc05-490a099b7477 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=6a80c579-20d5-46c7-8c07-6438f1367afd HTTP/1.1" status: 200 len: 853 time: 0.0691576 May 24 11:21:19.697415 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-023028ac-49b5-436e-bd3d-c55713232cde req-92d13d28-a040-435d-b0fb-94dd912f1d31 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:19.699237 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-023028ac-49b5-436e-bd3d-c55713232cde req-92d13d28-a040-435d-b0fb-94dd912f1d31 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0944142 May 24 11:21:19.776228 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:19.794811 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-4a9b5196-12e7-439d-827b-50600306f90f None None] Provisioning for port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:21:19.801517 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-4a9b5196-12e7-439d-827b-50600306f90f None None] Provisioning complete for port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:21:19.802005 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-4a9b5196-12e7-439d-827b-50600306f90f None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:19.860197 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-4a9b5196-12e7-439d-827b-50600306f90f None None] Port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:21:19.861814 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-727a12e9-d4be-415d-95c2-82edaf10ae8f req-c1d5ac21-6448-445a-8c53-5430782696e5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:19.862276 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-727a12e9-d4be-415d-95c2-82edaf10ae8f req-c1d5ac21-6448-445a-8c53-5430782696e5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Synchronizing usage tracker for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:21:19.866948 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-023028ac-49b5-436e-bd3d-c55713232cde req-838a0317-d5c7-4f29-8309-05897e963b51 service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1570299 May 24 11:21:19.890572 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-727a12e9-d4be-415d-95c2-82edaf10ae8f req-c1d5ac21-6448-445a-8c53-5430782696e5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Unset dirty status for tenant:b303f83a84794e5ba6a1bab9c76b429c on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:21:19.891891 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-727a12e9-d4be-415d-95c2-82edaf10ae8f req-c1d5ac21-6448-445a-8c53-5430782696e5 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 931 time: 0.2410138 May 24 11:21:20.007395 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-023028ac-49b5-436e-bd3d-c55713232cde req-21c5c67b-5ae9-4b16-8bea-efaa142b4219 service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1336503 May 24 11:21:20.071932 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-727a12e9-d4be-415d-95c2-82edaf10ae8f req-674d8aba-bf3d-4ea5-ae97-d3a20127105f tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1422799 May 24 11:21:20.225963 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 {{(pid=91820) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:21:20.233885 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:20.248196 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.provisioning_blocks [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Transition to ACTIVE for port object 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 will not be triggered until provisioned by entity L2. {{(pid=91820) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:21:20.252916 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:20.256541 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0b7f2485-d5bd-405f-8ce7-6fe309212805 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:20.410600 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Pushing event updated for resources: {'Port': ['ID=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6,revision_number=2']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:20.418838 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.162s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:20.619919 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Attempting to bind port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:21:20.620619 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Attempting to bind port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:21:20.620619 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Attempting to bind port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 on network 1d92641a-d8f9-4ed6-9abd-f464bb26811d {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:21:20.625312 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 on network 1d92641a-d8f9-4ed6-9abd-f464bb26811d not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:21:20.625624 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Attempting to bind port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 on network 1d92641a-d8f9-4ed6-9abd-f464bb26811d {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:21:20.631007 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 20, 52), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:21:20.631335 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Checking segment: {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:21:20.631714 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Bound using segment: {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'} {{(pid=91820) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:21:20.632126 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.managers [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Bound port: 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'bbef7673-75cf-415f-81a6-941c270eac8b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 518, 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d'}}] {{(pid=91820) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:21:20.632463 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:20.725243 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] For port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6, host hv-cinder-78409, cleared binding levels {{(pid=91820) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:21:20.728948 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] For port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91820) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:21:20.734392 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535262834'] for port, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:20.738715 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-2db77fa0-965f-4b49-9c4c-508051af2f6b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:20.800403 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:20.801316 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:20.802892 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] 10.222.0.93 "PUT /v2.0/ports/6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 HTTP/1.1" status: 200 len: 1093 time: 1.2187138 May 24 11:21:20.803372 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'tag': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:21:20.829051 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Pushing event updated for resources: {'Port': ['ID=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6,revision_number=3']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:20.834435 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:20.890742 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'tag': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6', 'status': 'completed', 'code': 200} May 24 11:21:20.935887 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a53749e-f836-45b3-8584-233dfdcce69f req-52732b91-ca3f-4d74-a6ea-f8e634970deb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:20.937431 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-52732b91-ca3f-4d74-a6ea-f8e634970deb service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=b303f83a84794e5ba6a1bab9c76b429c&device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 1096 time: 0.0869613 May 24 11:21:21.013291 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-b8b42e7e-168e-494d-a5be-54ddcdc74320 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 HTTP/1.1" status: 200 len: 217 time: 0.0670443 May 24 11:21:21.080980 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a53749e-f836-45b3-8584-233dfdcce69f req-f92068d1-1053-4d29-9493-8f537cb2fbcd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:21.082389 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-f92068d1-1053-4d29-9493-8f537cb2fbcd service neutron] 10.222.0.93 "GET /v2.0/subnets?id=2fc4be2e-875c-4d10-b857-a48f90c53cf2 HTTP/1.1" status: 200 len: 850 time: 0.0613790 May 24 11:21:21.164152 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a53749e-f836-45b3-8584-233dfdcce69f req-20e1cfcc-e378-49b9-9126-bd855f3a749c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:21.165688 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-20e1cfcc-e378-49b9-9126-bd855f3a749c service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=1d92641a-d8f9-4ed6-9abd-f464bb26811d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0756481 May 24 11:21:21.317047 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-26c8fe9d-bacd-443e-a168-ea80a9554608 req-7d76872a-5285-4041-9239-29379e1c93e1 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:21.317889 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-26c8fe9d-bacd-443e-a168-ea80a9554608 req-7d76872a-5285-4041-9239-29379e1c93e1 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 943 time: 0.1194351 May 24 11:21:21.330133 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-e8292f12-1c34-4906-8e82-8ffabf373e00 service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1546443 May 24 11:21:21.365979 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-26c8fe9d-bacd-443e-a168-ea80a9554608 req-9bb90e70-eaca-4004-adb6-6cfff8975131 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0416095 May 24 11:21:21.515181 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0a53749e-f836-45b3-8584-233dfdcce69f req-580edce2-2c73-45fa-ae64-b6107fa1e13c service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1726923 May 24 11:21:21.645314 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ececc0c-6339-46e1-ba47-727c31e7172a req-7ce3759e-c0e9-4601-887f-fe9bf901917b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:21.646830 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4ececc0c-6339-46e1-ba47-727c31e7172a req-7ce3759e-c0e9-4601-887f-fe9bf901917b service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=b303f83a84794e5ba6a1bab9c76b429c&device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 1096 time: 0.0719972 May 24 11:21:21.790462 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ececc0c-6339-46e1-ba47-727c31e7172a req-cd4ddcb6-cddc-4556-b367-31e806e254eb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:21.792285 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4ececc0c-6339-46e1-ba47-727c31e7172a req-cd4ddcb6-cddc-4556-b367-31e806e254eb service neutron] 10.222.0.93 "GET /v2.0/networks?id=1d92641a-d8f9-4ed6-9abd-f464bb26811d HTTP/1.1" status: 200 len: 912 time: 0.1385446 May 24 11:21:21.870311 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4ececc0c-6339-46e1-ba47-727c31e7172a req-33878272-5f35-46f7-bf2f-4a902dd5cabb service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 HTTP/1.1" status: 200 len: 217 time: 0.0679896 May 24 11:21:21.938437 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ececc0c-6339-46e1-ba47-727c31e7172a req-ae7fc0bb-71b2-4644-ac14-c209837de1a5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:21.939817 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4ececc0c-6339-46e1-ba47-727c31e7172a req-ae7fc0bb-71b2-4644-ac14-c209837de1a5 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=2fc4be2e-875c-4d10-b857-a48f90c53cf2 HTTP/1.1" status: 200 len: 850 time: 0.0595400 May 24 11:21:22.009819 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:22.022126 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ececc0c-6339-46e1-ba47-727c31e7172a req-c9f8084f-fe4f-415b-b826-f98d4d6dba05 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:22.023594 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4ececc0c-6339-46e1-ba47-727c31e7172a req-c9f8084f-fe4f-415b-b826-f98d4d6dba05 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=1d92641a-d8f9-4ed6-9abd-f464bb26811d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0767140 May 24 11:21:22.163478 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4ececc0c-6339-46e1-ba47-727c31e7172a req-b0016e22-c5bd-4e66-804b-ac2b34e620ca service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1315522 May 24 11:21:22.298191 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4ececc0c-6339-46e1-ba47-727c31e7172a req-208a8381-63dc-426d-87da-9968d582549d service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1269743 May 24 11:21:22.587140 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aca5e196-f805-492f-a010-881e0ca967b5 req-6af39b49-7bee-4a44-9dd9-730a0d81cc6f tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:22.589373 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-aca5e196-f805-492f-a010-881e0ca967b5 req-6af39b49-7bee-4a44-9dd9-730a0d81cc6f tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 943 time: 0.1140504 May 24 11:21:22.663886 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-aca5e196-f805-492f-a010-881e0ca967b5 req-132b6a91-622d-42c0-8cff-7fe147b94d6e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0670528 May 24 11:21:23.412908 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-06800255-ba72-467d-8987-e88e703d22f3 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:23.853280 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1c56b5d6-d593-4500-b28e-0dc6ae4bf6d3 req-01ebdff8-5e41-4947-b498-c80cc25f721c tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:23.854702 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1c56b5d6-d593-4500-b28e-0dc6ae4bf6d3 req-01ebdff8-5e41-4947-b498-c80cc25f721c tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 943 time: 0.0819151 May 24 11:21:23.901016 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1c56b5d6-d593-4500-b28e-0dc6ae4bf6d3 req-e6b6bf84-735b-4422-ac25-06545e8d3709 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0406320 May 24 11:21:25.090313 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8bff9d68-135e-461d-94ba-6afb1cc09401 req-2b3d924c-3df6-4c3d-bde1-2438000a4281 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:25.091663 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8bff9d68-135e-461d-94ba-6afb1cc09401 req-2b3d924c-3df6-4c3d-bde1-2438000a4281 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 943 time: 0.0842376 May 24 11:21:25.213948 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8bff9d68-135e-461d-94ba-6afb1cc09401 req-462cd201-7140-486d-9a94-0f8331d7a449 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1163990 May 24 11:21:25.744846 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:25.788154 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:21:25.892162 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3276812a-dc89-4331-98b4-b4fbe5b9c28b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:25.937586 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:25.951732 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'port_id': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6', 'mac_address': 'fa:16:3e:57:6c:e2', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 518, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.5'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:21:26.012197 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:26.067044 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:21:26.083984 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:26.087941 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:26.091491 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:26.155747 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:26.162701 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:26.163045 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:26.163379 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:26.163709 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:26.164038 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:26.164358 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:26.164676 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:26.164996 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:26.165368 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:26.165823 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:26.166222 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:26.166541 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:21:26.170704 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:26.353382 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:21:26.395998 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08fb92b1-325c-43f8-b2bb-d9bcd600b188 req-258c793a-39d9-453f-9884-7b5367552db6 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:26.397468 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-08fb92b1-325c-43f8-b2bb-d9bcd600b188 req-258c793a-39d9-453f-9884-7b5367552db6 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 944 time: 0.0813041 May 24 11:21:26.444676 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-08fb92b1-325c-43f8-b2bb-d9bcd600b188 req-737969cb-347c-415e-aedc-0d612a70fb68 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0416048 May 24 11:21:26.973921 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:27.635041 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b82b4e52-f9f8-41ee-939d-f9f8c122da0a req-a395e929-ca14-4719-b442-bc23e6e59eeb tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:27.636584 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b82b4e52-f9f8-41ee-939d-f9f8c122da0a req-a395e929-ca14-4719-b442-bc23e6e59eeb tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 944 time: 0.0824974 May 24 11:21:27.689702 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b82b4e52-f9f8-41ee-939d-f9f8c122da0a req-f600be29-d066-43c9-8a7e-285a29ae71ef tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0474164 May 24 11:21:27.805176 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:21:27.871429 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:21:27.876100 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:21:27.876552 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:27.995315 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:28.021065 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:21:28.031524 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:28.035730 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:28.037545 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:28.069950 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6', 'code': 200} May 24 11:21:28.114734 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:28.117911 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:28.498650 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f6cebc29-333a-412f-a937-2f8576b1e081 req-a0804f8c-dd02-4b0e-936c-2d3c1f6033ea service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:28.500278 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f6cebc29-333a-412f-a937-2f8576b1e081 req-a0804f8c-dd02-4b0e-936c-2d3c1f6033ea service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 1098 time: 0.2777214 May 24 11:21:28.544323 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f6cebc29-333a-412f-a937-2f8576b1e081 req-316fc6d6-d17b-48de-95b5-8759d4e7fa3e service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0375473 May 24 11:21:28.884382 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3f080875-03f9-4a13-b384-23342ffd0fb2 req-056bdb64-c492-4752-bfb2-d4b8f5ab9b53 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:28.886425 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3f080875-03f9-4a13-b384-23342ffd0fb2 req-056bdb64-c492-4752-bfb2-d4b8f5ab9b53 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 945 time: 0.0898838 May 24 11:21:28.918879 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:28.935241 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3f080875-03f9-4a13-b384-23342ffd0fb2 req-9366114b-037a-4e31-b352-f2dfd7d1843a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0429933 May 24 11:21:30.118823 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cbfb2628-92dd-4af0-80fb-6f3c54287686 req-efbd6d5e-e1be-4563-8791-b36c7f8b119f tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:30.120928 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cbfb2628-92dd-4af0-80fb-6f3c54287686 req-efbd6d5e-e1be-4563-8791-b36c7f8b119f tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 945 time: 0.0812051 May 24 11:21:30.169474 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cbfb2628-92dd-4af0-80fb-6f3c54287686 req-be3b560d-66b5-4611-8cbe-80c80ac1b55c tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0415847 May 24 11:21:30.364770 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72900a90-7764-4971-aaed-b6df776c9998 req-3c64c453-b551-49c9-bf01-baa401b305d7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:30.366258 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-72900a90-7764-4971-aaed-b6df776c9998 req-3c64c453-b551-49c9-bf01-baa401b305d7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.0819590 May 24 11:21:30.369450 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e4cbbba4-9c98-4ed9-91ef-5880d7c66682 req-5bf2afa2-a89e-44ff-b757-e85c9812baaa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:30.370878 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e4cbbba4-9c98-4ed9-91ef-5880d7c66682 req-5bf2afa2-a89e-44ff-b757-e85c9812baaa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 945 time: 0.0905197 May 24 11:21:30.379294 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 58928) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:30.384568 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-deec2544-f57e-4317-aaa8-10f7d569be05 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Request body: {'floatingip': {'port_id': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:21:30.411427 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-72900a90-7764-4971-aaed-b6df776c9998 req-4f901f4b-d6de-470d-9448-9177b984db1e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393510 May 24 11:21:30.463136 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-deec2544-f57e-4317-aaa8-10f7d569be05 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, before_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:31.058799 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-deec2544-f57e-4317-aaa8-10f7d569be05 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:31.062564 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-deec2544-f57e-4317-aaa8-10f7d569be05 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:31.234994 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.l3_db [None req-deec2544-f57e-4317-aaa8-10f7d569be05 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Floating IP 5283da39-39a3-4d0a-887a-08f5fdb56c35 associated. External IP: 172.27.0.194, port: 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6. May 24 11:21:31.754029 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28dc0366-b7f2-4d9e-92cc-39d4cbd9449e req-178d089f-5efa-4f28-91f5-700004497938 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:31.755488 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-28dc0366-b7f2-4d9e-92cc-39d4cbd9449e req-178d089f-5efa-4f28-91f5-700004497938 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1957724 May 24 11:21:31.779632 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-deec2544-f57e-4317-aaa8-10f7d569be05 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Router f76568b3-17ef-487b-beeb-3f592a58e138 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:21:31.870961 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7367a6ea-7d68-440e-9413-5d583b312bcf req-8a27b471-e406-47a7-9e86-59b328d14459 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-1726653752&fields=id&tenant_id=788050c389224d17b163f24bb67902bf HTTP/1.1" status: 200 len: 267 time: 0.0402956 May 24 11:21:31.938314 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-28dc0366-b7f2-4d9e-92cc-39d4cbd9449e req-ed03f4be-c435-4b33-aaa5-1a5b5235d139 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1757500 May 24 11:21:31.968246 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7367a6ea-7d68-440e-9413-5d583b312bcf req-6fb3cefa-47c5-4129-bb72-00536d5da6f7 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:31.969750 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7367a6ea-7d68-440e-9413-5d583b312bcf req-6fb3cefa-47c5-4129-bb72-00536d5da6f7 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 984 time: 0.0931530 May 24 11:21:31.980322 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-7367a6ea-7d68-440e-9413-5d583b312bcf req-4579ebdd-a23e-4b77-ac5a-26d5b21d74e0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Request body: {'port': {'security_groups': ['29c10cc8-55e7-4791-971b-485a28ba237e']}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:21:31.996723 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-deec2544-f57e-4317-aaa8-10f7d569be05 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:21:32.001723 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-deec2544-f57e-4317-aaa8-10f7d569be05 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:32.007153 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0a9d3699-41b9-4c03-8dd8-ea2527443227 None None] Sync routers for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:21:32.017410 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0a9d3699-41b9-4c03-8dd8-ea2527443227 None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:21:32.018035 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-0a9d3699-41b9-4c03-8dd8-ea2527443227 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:21:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:32.055982 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-deec2544-f57e-4317-aaa8-10f7d569be05 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "PUT /v2.0/floatingips/5283da39-39a3-4d0a-887a-08f5fdb56c35 HTTP/1.1" status: 200 len: 991 time: 1.6755750 May 24 11:21:32.056860 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'tag': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:21:32.111322 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-7367a6ea-7d68-440e-9413-5d583b312bcf req-4579ebdd-a23e-4b77-ac5a-26d5b21d74e0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:32.210666 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-0a9d3699-41b9-4c03-8dd8-ea2527443227 None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:21:32.360815 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'tag': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6', 'status': 'completed', 'code': 200} May 24 11:21:32.381185 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-7367a6ea-7d68-440e-9413-5d583b312bcf req-4579ebdd-a23e-4b77-ac5a-26d5b21d74e0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] For port e0dbce52-01cb-4042-b340-ea0a6b7da010, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:32.406639 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-7367a6ea-7d68-440e-9413-5d583b312bcf req-4579ebdd-a23e-4b77-ac5a-26d5b21d74e0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:32.411768 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-7367a6ea-7d68-440e-9413-5d583b312bcf req-4579ebdd-a23e-4b77-ac5a-26d5b21d74e0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:32.418891 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:32.451782 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['e0dbce52-01cb-4042-b340-ea0a6b7da010'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:21:32.456351 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a435cfdd-29cc-41ff-926a-8c821cbb094f req-836ce6c6-2bb6-4bef-987e-13e4b9d2b347 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:32.458456 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a435cfdd-29cc-41ff-926a-8c821cbb094f req-836ce6c6-2bb6-4bef-987e-13e4b9d2b347 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=b303f83a84794e5ba6a1bab9c76b429c&device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 1098 time: 0.0759110 May 24 11:21:32.489743 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7367a6ea-7d68-440e-9413-5d583b312bcf req-4579ebdd-a23e-4b77-ac5a-26d5b21d74e0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:32.490375 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-7367a6ea-7d68-440e-9413-5d583b312bcf req-4579ebdd-a23e-4b77-ac5a-26d5b21d74e0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:32.491762 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7367a6ea-7d68-440e-9413-5d583b312bcf req-4579ebdd-a23e-4b77-ac5a-26d5b21d74e0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "PUT /v2.0/ports/e0dbce52-01cb-4042-b340-ea0a6b7da010 HTTP/1.1" status: 200 len: 942 time: 0.5159950 May 24 11:21:32.492276 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:21:32.511663 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58942) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:32.633235 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a435cfdd-29cc-41ff-926a-8c821cbb094f req-154f3f62-796b-46aa-a898-a4b0c33a1a69 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:32.634956 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a435cfdd-29cc-41ff-926a-8c821cbb094f req-154f3f62-796b-46aa-a898-a4b0c33a1a69 service neutron] 10.222.0.93 "GET /v2.0/networks?id=1d92641a-d8f9-4ed6-9abd-f464bb26811d HTTP/1.1" status: 200 len: 912 time: 0.1704874 May 24 11:21:32.640681 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'status': 'completed', 'code': 200} May 24 11:21:32.664709 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4579ebdd-a23e-4b77-ac5a-26d5b21d74e0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event updated for resources: {'Port': ['ID=e0dbce52-01cb-4042-b340-ea0a6b7da010,revision_number=7']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:32.676786 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4579ebdd-a23e-4b77-ac5a-26d5b21d74e0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.258s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:32.679023 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['e0dbce52-01cb-4042-b340-ea0a6b7da010'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:21:32.692809 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3ec168a0-5218-4a9c-a4cc-fb936ace2b45 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:32.736875 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3ec168a0-5218-4a9c-a4cc-fb936ace2b45 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:32.753117 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71d6ef68-e72d-4ec5-a2ed-747e9b8d04e9 req-2ed1ac43-a567-4b2a-bd1c-5f54cd40afeb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:32.754608 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71d6ef68-e72d-4ec5-a2ed-747e9b8d04e9 req-2ed1ac43-a567-4b2a-bd1c-5f54cd40afeb service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=788050c389224d17b163f24bb67902bf&device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 1098 time: 0.1520569 May 24 11:21:32.755872 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a435cfdd-29cc-41ff-926a-8c821cbb094f req-130b25ef-4efa-4e01-91a8-c498ec84c7c6 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 HTTP/1.1" status: 200 len: 999 time: 0.1099234 May 24 11:21:32.829057 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a435cfdd-29cc-41ff-926a-8c821cbb094f req-8d382d32-2257-4ab2-a993-05a621bb776b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:32.830425 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a435cfdd-29cc-41ff-926a-8c821cbb094f req-8d382d32-2257-4ab2-a993-05a621bb776b service neutron] 10.222.0.93 "GET /v2.0/subnets?id=2fc4be2e-875c-4d10-b857-a48f90c53cf2 HTTP/1.1" status: 200 len: 850 time: 0.0643694 May 24 11:21:32.866841 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-0a9d3699-41b9-4c03-8dd8-ea2527443227 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '5283da39-39a3-4d0a-887a-08f5fdb56c35', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'floating_ip_address': '172.27.0.194', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'port_id': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 525, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'mac_address': 'fa:16:3e:57:6c:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:21:15Z', 'updated_at': '2021-05-24T11:21:30Z', 'revision_number': 1, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:17Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:32.890491 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0a9d3699-41b9-4c03-8dd8-ea2527443227 None None] Checking router: f76568b3-17ef-487b-beeb-3f592a58e138 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:21:32.892482 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-c423003c-cd76-4d40-92a2-f6cae8f46f97 None None] Provisioning complete for port e0dbce52-01cb-4042-b340-ea0a6b7da010 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:21:32.892867 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c423003c-cd76-4d40-92a2-f6cae8f46f97 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:32.911127 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71d6ef68-e72d-4ec5-a2ed-747e9b8d04e9 req-fcece42e-4bd5-41e4-9040-dc0c010e02f8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:32.912446 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a435cfdd-29cc-41ff-926a-8c821cbb094f req-f6eea825-b925-4759-abc1-1e2c03a4c782 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:32.912589 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71d6ef68-e72d-4ec5-a2ed-747e9b8d04e9 req-fcece42e-4bd5-41e4-9040-dc0c010e02f8 service neutron] 10.222.0.93 "GET /v2.0/networks?id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 HTTP/1.1" status: 200 len: 915 time: 0.1473074 May 24 11:21:32.913987 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a435cfdd-29cc-41ff-926a-8c821cbb094f req-f6eea825-b925-4759-abc1-1e2c03a4c782 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=1d92641a-d8f9-4ed6-9abd-f464bb26811d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0773244 May 24 11:21:32.923286 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3ec168a0-5218-4a9c-a4cc-fb936ace2b45 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:32.929904 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3880df7f-8735-4ad5-8097-dc6c136d9b9f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:32.976668 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0a9d3699-41b9-4c03-8dd8-ea2527443227 None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:21:32.978151 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-0a9d3699-41b9-4c03-8dd8-ea2527443227 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:21:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:33.048870 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71d6ef68-e72d-4ec5-a2ed-747e9b8d04e9 req-a96e27a3-e066-4618-824e-4b6b28727af7 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010 HTTP/1.1" status: 200 len: 1001 time: 0.1202557 May 24 11:21:33.055411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3ec168a0-5218-4a9c-a4cc-fb936ace2b45 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=9286a221-16dd-4047-bf8c-6373837405d0,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:33.058564 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3ec168a0-5218-4a9c-a4cc-fb936ace2b45 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:33.083599 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a435cfdd-29cc-41ff-926a-8c821cbb094f req-12c771f9-7669-4710-824b-c2d44c162c34 service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1586311 May 24 11:21:33.146858 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3ec168a0-5218-4a9c-a4cc-fb936ace2b45 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:33.147983 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-3ec168a0-5218-4a9c-a4cc-fb936ace2b45 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "DELETE /v2.0/security-groups/9286a221-16dd-4047-bf8c-6373837405d0 HTTP/1.1" status: 204 len: 168 time: 0.6328220 May 24 11:21:33.157671 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 58948) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:33.243908 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a435cfdd-29cc-41ff-926a-8c821cbb094f req-e05e27e6-9d91-486b-8e51-05c1d488677d service neutron] 10.222.0.93 "GET /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1538904 May 24 11:21:33.294401 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71d6ef68-e72d-4ec5-a2ed-747e9b8d04e9 req-30cfc71e-6723-474a-b55f-fe3c93214cfa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:33.295830 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71d6ef68-e72d-4ec5-a2ed-747e9b8d04e9 req-30cfc71e-6723-474a-b55f-fe3c93214cfa service neutron] 10.222.0.93 "GET /v2.0/subnets?id=6a80c579-20d5-46c7-8c07-6438f1367afd HTTP/1.1" status: 200 len: 853 time: 0.2352488 May 24 11:21:33.356874 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-0a9d3699-41b9-4c03-8dd8-ea2527443227 None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:21:33.612338 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-89f2a643-e104-4382-8081-62adc5ee8272 req-8bda1090-7218-473d-b520-f326ef23961f tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:33.613832 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-89f2a643-e104-4382-8081-62adc5ee8272 req-8bda1090-7218-473d-b520-f326ef23961f tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.5146403 May 24 11:21:33.659956 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71d6ef68-e72d-4ec5-a2ed-747e9b8d04e9 req-36b77a6b-9281-4ae2-a4a7-212b96eb1f0a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:33.661644 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71d6ef68-e72d-4ec5-a2ed-747e9b8d04e9 req-36b77a6b-9281-4ae2-a4a7-212b96eb1f0a service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.3570151 May 24 11:21:33.744191 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-0a9d3699-41b9-4c03-8dd8-ea2527443227 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '5283da39-39a3-4d0a-887a-08f5fdb56c35', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'floating_ip_address': '172.27.0.194', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'port_id': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 525, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'mac_address': 'fa:16:3e:57:6c:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:21:15Z', 'updated_at': '2021-05-24T11:21:30Z', 'revision_number': 1, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:17Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:33.748666 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0a9d3699-41b9-4c03-8dd8-ea2527443227 None None] The sync data for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi is: [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '5283da39-39a3-4d0a-887a-08f5fdb56c35', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'floating_ip_address': '172.27.0.194', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'port_id': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 525, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'mac_address': 'fa:16:3e:57:6c:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:21:15Z', 'updated_at': '2021-05-24T11:21:30Z', 'revision_number': 1, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:17Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:21:33.756637 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-89f2a643-e104-4382-8081-62adc5ee8272 req-58bed185-2d6e-42ad-b807-8ebc95b736f9 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1312771 May 24 11:21:33.771590 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Deleting port 68eb8843-6cde-414a-9854-2653eea784b1 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:21:33.772157 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:33.915180 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71d6ef68-e72d-4ec5-a2ed-747e9b8d04e9 req-90f81950-84a3-44b4-845c-a9595a1219f8 service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2432666 May 24 11:21:33.957292 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bc72739e-b634-4382-af59-aa44548f9d4c None None] New status for floating IP 5283da39-39a3-4d0a-887a-08f5fdb56c35: ACTIVE {{(pid=91822) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:21:34.001259 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:34.001590 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:34.136923 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Calling delete_port for 68eb8843-6cde-414a-9854-2653eea784b1 owned by network:floatingip {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:21:34.200758 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-71d6ef68-e72d-4ec5-a2ed-747e9b8d04e9 req-e14d48a1-3a39-4019-9d66-399639a45ff3 service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2710211 May 24 11:21:34.307878 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:34.311344 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:34.356688 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event deleted for resources: {'Port': ['ID=68eb8843-6cde-414a-9854-2653eea784b1,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:34.381677 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:34.448615 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks [] for floatingip, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:34.448950 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Floating IP 02886571-aeff-4f23-8fb1-dd7a8651636b disassociated (deleted). External IP: 172.27.0.236, port: e0dbce52-01cb-4042-b340-ea0a6b7da010. May 24 11:21:34.819448 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Router 684315b8-ba5e-4d23-8963-64aca3b067cb has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:21:35.192523 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:21:35.202079 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ba8a263c-0536-486f-be9d-6281c3f6940d None None] Sync routers for ids ['684315b8-ba5e-4d23-8963-64aca3b067cb'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:21:35.207038 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0100e615-97e9-451b-bcf3-b0475b1e6efe req-c85bbee3-b699-4209-9570-91aa9cf0db39 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:35.208483 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0100e615-97e9-451b-bcf3-b0475b1e6efe req-c85bbee3-b699-4209-9570-91aa9cf0db39 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.3327191 May 24 11:21:35.211459 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ba8a263c-0536-486f-be9d-6281c3f6940d None None] Routers {'684315b8-ba5e-4d23-8963-64aca3b067cb'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:21:35.212044 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-ba8a263c-0536-486f-be9d-6281c3f6940d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:21:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['684315b8-ba5e-4d23-8963-64aca3b067cb'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:35.240648 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:35.301808 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0100e615-97e9-451b-bcf3-b0475b1e6efe req-85f91c54-145e-4cee-8f23-2d5add8a88a7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0867629 May 24 11:21:35.314188 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "DELETE /v2.0/floatingips/02886571-aeff-4f23-8fb1-dd7a8651636b HTTP/1.1" status: 204 len: 168 time: 2.1529460 May 24 11:21:35.314188 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:21:35.413006 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-ba8a263c-0536-486f-be9d-6281c3f6940d None None] Set router 684315b8-ba5e-4d23-8963-64aca3b067cb gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:21:35.512300 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'status': 'completed', 'code': 200} May 24 11:21:35.637528 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-52430291-d3eb-4778-b103-c53efc18e0c1 req-c33e4fef-fbd5-4338-b4a4-71306e97f155 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:35.639026 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-52430291-d3eb-4778-b103-c53efc18e0c1 req-c33e4fef-fbd5-4338-b4a4-71306e97f155 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=788050c389224d17b163f24bb67902bf&device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 1098 time: 0.1020372 May 24 11:21:35.782620 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-ba8a263c-0536-486f-be9d-6281c3f6940d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:18:39Z', 'revision_number': 4, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '90af9233-b13f-40e5-a0f8-872fe40ddfed', 'name': '', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'mac_address': 'fa:16:3e:ab:4f:f6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_interface', 'standard_attr_id': 356, 'fixed_ips': [{'subnet_id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:38Z', 'updated_at': '2021-05-24T11:18:52Z', 'revision_number': 5, 'project_id': '788050c389224d17b163f24bb67902bf', 'subnets': [{'id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:35.786634 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ba8a263c-0536-486f-be9d-6281c3f6940d None None] Checking router: 684315b8-ba5e-4d23-8963-64aca3b067cb for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:21:35.796645 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ba8a263c-0536-486f-be9d-6281c3f6940d None None] Routers {'684315b8-ba5e-4d23-8963-64aca3b067cb'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:21:35.797354 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-ba8a263c-0536-486f-be9d-6281c3f6940d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:21:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['684315b8-ba5e-4d23-8963-64aca3b067cb'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:35.800436 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-52430291-d3eb-4778-b103-c53efc18e0c1 req-136567a7-e705-4031-891f-035eaff936aa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:35.800579 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-52430291-d3eb-4778-b103-c53efc18e0c1 req-136567a7-e705-4031-891f-035eaff936aa service neutron] 10.222.0.93 "GET /v2.0/networks?id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 HTTP/1.1" status: 200 len: 915 time: 0.1556118 May 24 11:21:35.888235 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-52430291-d3eb-4778-b103-c53efc18e0c1 req-3f8c97ca-f892-4f8f-aa38-802b75fe11bc service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010 HTTP/1.1" status: 200 len: 217 time: 0.0767901 May 24 11:21:35.952639 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-52430291-d3eb-4778-b103-c53efc18e0c1 req-5a489710-8c89-4c75-b5d5-9e233ad607ca service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:35.954116 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-52430291-d3eb-4778-b103-c53efc18e0c1 req-5a489710-8c89-4c75-b5d5-9e233ad607ca service neutron] 10.222.0.93 "GET /v2.0/subnets?id=6a80c579-20d5-46c7-8c07-6438f1367afd HTTP/1.1" status: 200 len: 853 time: 0.0590165 May 24 11:21:35.975811 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-ba8a263c-0536-486f-be9d-6281c3f6940d None None] Set router 684315b8-ba5e-4d23-8963-64aca3b067cb gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:21:36.032881 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-52430291-d3eb-4778-b103-c53efc18e0c1 req-21096e33-cae2-4531-b67a-c732bc3892c3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:36.034400 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-52430291-d3eb-4778-b103-c53efc18e0c1 req-21096e33-cae2-4531-b67a-c732bc3892c3 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0740128 May 24 11:21:36.175857 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-52430291-d3eb-4778-b103-c53efc18e0c1 req-6467ebc1-fd1d-4b9b-b1ad-3ef11cd0b70c service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1338010 May 24 11:21:36.330691 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-52430291-d3eb-4778-b103-c53efc18e0c1 req-6d281a12-913e-4b34-84c2-d50315566b99 service neutron] 10.222.0.93 "GET /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1474972 May 24 11:21:36.391046 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-ba8a263c-0536-486f-be9d-6281c3f6940d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:18:39Z', 'revision_number': 4, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '90af9233-b13f-40e5-a0f8-872fe40ddfed', 'name': '', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'mac_address': 'fa:16:3e:ab:4f:f6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_interface', 'standard_attr_id': 356, 'fixed_ips': [{'subnet_id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:38Z', 'updated_at': '2021-05-24T11:18:52Z', 'revision_number': 5, 'project_id': '788050c389224d17b163f24bb67902bf', 'subnets': [{'id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:36.395480 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ba8a263c-0536-486f-be9d-6281c3f6940d None None] The sync data for ids ['684315b8-ba5e-4d23-8963-64aca3b067cb'] in dv-cinder-784092-9-iscsi is: [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:18:39Z', 'revision_number': 4, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '90af9233-b13f-40e5-a0f8-872fe40ddfed', 'name': '', 'network_id': '0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'mac_address': 'fa:16:3e:ab:4f:f6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_interface', 'standard_attr_id': 356, 'fixed_ips': [{'subnet_id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:38Z', 'updated_at': '2021-05-24T11:18:52Z', 'revision_number': 5, 'project_id': '788050c389224d17b163f24bb67902bf', 'subnets': [{'id': '6a80c579-20d5-46c7-8c07-6438f1367afd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:21:36.556418 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3df457c7-617f-4e03-8815-549f368fe812 req-ebeff932-0ad7-442e-8e5d-480fcf00b331 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:36.558141 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3df457c7-617f-4e03-8815-549f368fe812 req-ebeff932-0ad7-442e-8e5d-480fcf00b331 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1118665 May 24 11:21:36.620323 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3df457c7-617f-4e03-8815-549f368fe812 req-bee15e52-c4ba-4eff-85c2-d2e4f7dae54a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0507870 May 24 11:21:36.673706 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-30a04a7b-755d-4e66-a92b-8993bfff1388 None None] New status for floating IP 02886571-aeff-4f23-8fb1-dd7a8651636b: DOWN {{(pid=91822) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:21:37.835163 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f9516c0-bbfc-45ab-9764-dcb910d44b0f req-7e2fbeda-bf3c-4e37-b565-d3c248081e4b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:37.836521 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4f9516c0-bbfc-45ab-9764-dcb910d44b0f req-7e2fbeda-bf3c-4e37-b565-d3c248081e4b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.0836585 May 24 11:21:37.886612 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4f9516c0-bbfc-45ab-9764-dcb910d44b0f req-bca9bb75-cb18-4f31-ac39-c0f4069dbc7e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0441842 May 24 11:21:38.770575 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-599f4be9-95ed-4742-a3c0-b9708bb19d13 req-2556b812-2803-482f-ae2d-e158c02e9b5a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:38.772053 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-599f4be9-95ed-4742-a3c0-b9708bb19d13 req-2556b812-2803-482f-ae2d-e158c02e9b5a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 945 time: 0.0865648 May 24 11:21:38.816816 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-599f4be9-95ed-4742-a3c0-b9708bb19d13 req-faa70a62-8f55-4464-a783-9bbcd821a120 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0384266 May 24 11:21:39.083708 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-668350eb-85f7-43b2-acbe-eb84a9100493 req-50eb7da4-5637-49e5-a0bd-906e14f0f566 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:39.085216 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-668350eb-85f7-43b2-acbe-eb84a9100493 req-50eb7da4-5637-49e5-a0bd-906e14f0f566 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.0827396 May 24 11:21:39.130600 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-668350eb-85f7-43b2-acbe-eb84a9100493 req-a70346a5-7f0a-4546-9456-dfa2eab4e80a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0392175 May 24 11:21:40.006645 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-53b66920-48f4-4ba4-9d8e-0b6d3ea5d885 req-9589c8f8-de89-4673-a0d3-520525b5e0fb tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:40.008136 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-53b66920-48f4-4ba4-9d8e-0b6d3ea5d885 req-9589c8f8-de89-4673-a0d3-520525b5e0fb tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 945 time: 0.0834353 May 24 11:21:40.057619 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-53b66920-48f4-4ba4-9d8e-0b6d3ea5d885 req-f3717f6e-5498-4e9a-a21a-590c7041c347 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/security-groups?id=29c10cc8-55e7-4791-971b-485a28ba237e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0432074 May 24 11:21:40.432134 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7a5f02cb-7da6-4cf8-a68e-9becd97937f6 req-76435617-d1a4-432c-afe5-39417e346169 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:40.433663 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7a5f02cb-7da6-4cf8-a68e-9becd97937f6 req-76435617-d1a4-432c-afe5-39417e346169 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1917653 May 24 11:21:40.480079 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7a5f02cb-7da6-4cf8-a68e-9becd97937f6 req-aefdfe1c-577f-41bc-b8ee-3070ca474e74 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0405684 May 24 11:21:40.535673 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-c4a7ac3e-81ef-4c5f-9fec-1d358e38fa3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:40.536361 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-c4a7ac3e-81ef-4c5f-9fec-1d358e38fa3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Synchronizing usage tracker for tenant:788050c389224d17b163f24bb67902bf on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:21:40.552766 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-c4a7ac3e-81ef-4c5f-9fec-1d358e38fa3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Unset dirty status for tenant:788050c389224d17b163f24bb67902bf on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:21:40.554249 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-c4a7ac3e-81ef-4c5f-9fec-1d358e38fa3a tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.93 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 945 time: 0.1056695 May 24 11:21:40.691990 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-7e1f88c0-ac85-4973-b705-44bbf4954b18 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Deleting port e0dbce52-01cb-4042-b340-ea0a6b7da010 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:21:40.692242 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-7e1f88c0-ac85-4973-b705-44bbf4954b18 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:40.910902 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-7e1f88c0-ac85-4973-b705-44bbf4954b18 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] For port e0dbce52-01cb-4042-b340-ea0a6b7da010, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=e0dbce52-01cb-4042-b340-ea0a6b7da010,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:40.911293 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-7e1f88c0-ac85-4973-b705-44bbf4954b18 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:41.035509 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-7e1f88c0-ac85-4973-b705-44bbf4954b18 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Calling delete_port for e0dbce52-01cb-4042-b340-ea0a6b7da010 owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:21:41.146828 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-7e1f88c0-ac85-4973-b705-44bbf4954b18 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:41.153272 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:41.158639 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['e0dbce52-01cb-4042-b340-ea0a6b7da010'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:21:41.237835 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7e1f88c0-ac85-4973-b705-44bbf4954b18 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Pushing event deleted for resources: {'Port': ['ID=e0dbce52-01cb-4042-b340-ea0a6b7da010,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:41.269529 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f876eb8e-d903-431e-b955-8f105c3f6714 req-24998517-3fcc-468e-84fc-c58b0ca18720 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.18 "GET /v2.0/ports?device_id=fa7a9207-daa9-4469-9eec-51039940736a HTTP/1.1" status: 200 len: 210 time: 0.0869982 May 24 11:21:41.270374 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-7e1f88c0-ac85-4973-b705-44bbf4954b18 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:41.319775 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-7e1f88c0-ac85-4973-b705-44bbf4954b18 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] No DVR routers for this DVR port e0dbce52-01cb-4042-b340-ea0a6b7da010 on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:21:41.322743 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-7e1f88c0-ac85-4973-b705-44bbf4954b18 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:41.325401 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'name': 'network-vif-deleted', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:21:41.341806 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5ffdcf58-9134-44de-bba3-89cd7ddac246 req-7e1f88c0-ac85-4973-b705-44bbf4954b18 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] 10.222.0.93 "DELETE /v2.0/ports/e0dbce52-01cb-4042-b340-ea0a6b7da010 HTTP/1.1" status: 204 len: 173 time: 0.7816875 May 24 11:21:41.375111 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'fa7a9207-daa9-4469-9eec-51039940736a', 'name': 'network-vif-deleted', 'tag': 'e0dbce52-01cb-4042-b340-ea0a6b7da010', 'status': 'completed', 'code': 200} May 24 11:21:41.707048 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eb582682-3ed8-423c-8987-2c5f4278357d req-4ae42eb9-a51f-48f2-ad9e-9d30de05176b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:41.708561 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-eb582682-3ed8-423c-8987-2c5f4278357d req-4ae42eb9-a51f-48f2-ad9e-9d30de05176b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1078699 May 24 11:21:41.755502 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-eb582682-3ed8-423c-8987-2c5f4278357d req-487f7b1b-161d-4d5b-876f-418bb520ff53 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0411603 May 24 11:21:43.007934 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e4ef05a8-c5d2-4848-9e68-facdedadaf70 req-e5bcc5e6-05c8-442e-98e6-50533bdd1d99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:43.009980 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e4ef05a8-c5d2-4848-9e68-facdedadaf70 req-e5bcc5e6-05c8-442e-98e6-50533bdd1d99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1152699 May 24 11:21:43.088989 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e4ef05a8-c5d2-4848-9e68-facdedadaf70 req-e9ad92ac-2cdc-468d-994e-acce0e187ac5 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0691361 May 24 11:21:43.168843 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59064) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:43.704891 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-687465'] for router_interface, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:44.107787 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] Deleting port 90af9233-b13f-40e5-a0f8-872fe40ddfed {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:21:44.108263 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:44.231840 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] For port 90af9233-b13f-40e5-a0f8-872fe40ddfed, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=90af9233-b13f-40e5-a0f8-872fe40ddfed,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:44.232549 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:44.282663 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-73d7bbe1-c47a-45e3-941e-f73ace1aeef9 req-3909c5a8-a2b4-436d-9f98-d498f7b93f36 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:44.284151 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-73d7bbe1-c47a-45e3-941e-f73ace1aeef9 req-3909c5a8-a2b4-436d-9f98-d498f7b93f36 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.0811436 May 24 11:21:44.341851 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-73d7bbe1-c47a-45e3-941e-f73ace1aeef9 req-37034267-224b-4512-ba8f-cdecffdcc8fd tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0516734 May 24 11:21:44.459144 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] Calling delete_port for 90af9233-b13f-40e5-a0f8-872fe40ddfed owned by network:router_interface {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:21:44.596577 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:44.599769 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ac9ebc5c-5602-4b2b-993b-a43590ced740 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:44.645414 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] Pushing event deleted for resources: {'Port': ['ID=90af9233-b13f-40e5-a0f8-872fe40ddfed,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:44.648611 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:44.915524 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-537767', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-685561'] for router_interface, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:45.277031 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] Router 684315b8-ba5e-4d23-8963-64aca3b067cb has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:21:45.469718 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:21:45.482079 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ac7fdd19-e2e5-4307-8e79-6e40411e6936 None None] Sync routers for ids ['684315b8-ba5e-4d23-8963-64aca3b067cb'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:21:45.483589 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] 10.222.0.18 "PUT /v2.0/routers/684315b8-ba5e-4d23-8963-64aca3b067cb/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.3130145 May 24 11:21:45.486155 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59076) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:45.495248 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ac7fdd19-e2e5-4307-8e79-6e40411e6936 None None] Routers {'684315b8-ba5e-4d23-8963-64aca3b067cb'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:21:45.495879 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-ac7fdd19-e2e5-4307-8e79-6e40411e6936 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:21:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['684315b8-ba5e-4d23-8963-64aca3b067cb'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:45.571607 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46064ac7-b523-44ef-9d05-08850c2eeb54 req-957fb541-5fe2-4eb9-b6f2-92622cc9b0ec tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:45.573917 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-46064ac7-b523-44ef-9d05-08850c2eeb54 req-957fb541-5fe2-4eb9-b6f2-92622cc9b0ec tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1151073 May 24 11:21:45.631547 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-46064ac7-b523-44ef-9d05-08850c2eeb54 req-500e3a79-bce9-4230-889a-977a78578233 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0508947 May 24 11:21:45.699099 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Notify callbacks [] for router, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:45.701006 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-ac7fdd19-e2e5-4307-8e79-6e40411e6936 None None] Set router 684315b8-ba5e-4d23-8963-64aca3b067cb gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:21:45.924285 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device e0dbce52-01cb-4042-b340-ea0a6b7da010 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:21:45.976594 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: e0dbce52-01cb-4042-b340-ea0a6b7da010 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:21:45.977499 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device e0dbce52-01cb-4042-b340-ea0a6b7da010 not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:21:45.997438 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:46.055790 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-ac7fdd19-e2e5-4307-8e79-6e40411e6936 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200'}], 'enable_snat': True}, 'gw_port_id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:21:44Z', 'revision_number': 5, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port': {'id': '54c19980-7452-40ac-b0f5-c89b800a3d24', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:8e:30:a4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.200', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:34Z', 'updated_at': '2021-05-24T11:18:41Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:46.059889 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ac7fdd19-e2e5-4307-8e79-6e40411e6936 None None] Checking router: 684315b8-ba5e-4d23-8963-64aca3b067cb for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:21:46.069254 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ac7fdd19-e2e5-4307-8e79-6e40411e6936 None None] Routers {'684315b8-ba5e-4d23-8963-64aca3b067cb'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:21:46.069877 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-ac7fdd19-e2e5-4307-8e79-6e40411e6936 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:21:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['684315b8-ba5e-4d23-8963-64aca3b067cb'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:46.079324 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Deleting port 54c19980-7452-40ac-b0f5-c89b800a3d24 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:21:46.079801 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:46.209335 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] For port 54c19980-7452-40ac-b0f5-c89b800a3d24, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=54c19980-7452-40ac-b0f5-c89b800a3d24,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:46.209685 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:46.225512 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-ac7fdd19-e2e5-4307-8e79-6e40411e6936 None None] Set router 684315b8-ba5e-4d23-8963-64aca3b067cb gateway port host: None {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:21:46.334965 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-ac7fdd19-e2e5-4307-8e79-6e40411e6936 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:21:45Z', 'revision_number': 6, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port_host': None}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:46.336376 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Calling delete_port for 54c19980-7452-40ac-b0f5-c89b800a3d24 owned by network:router_gateway {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:21:46.339195 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ac7fdd19-e2e5-4307-8e79-6e40411e6936 None None] The sync data for ids ['684315b8-ba5e-4d23-8963-64aca3b067cb'] in dv-cinder-784092-9-iscsi is: [{'id': '684315b8-ba5e-4d23-8963-64aca3b067cb', 'name': 'tempest-TestMinimumBasicScenario-953866096-router', 'tenant_id': '788050c389224d17b163f24bb67902bf', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:33Z', 'updated_at': '2021-05-24T11:21:45Z', 'revision_number': 6, 'project_id': '788050c389224d17b163f24bb67902bf', 'gw_port_host': None}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:21:46.483124 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:46.487539 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-350b5ffb-770e-48cd-8e38-75a5eca39983 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:46.533270 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Pushing event deleted for resources: {'Port': ['ID=54c19980-7452-40ac-b0f5-c89b800a3d24,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:46.566343 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:46.622973 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-686964'] for router_gateway, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:46.831976 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-684537', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4171'] for router, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:46.834448 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:46.855630 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bea3fab1-a680-4465-9509-5a73df01cc8d req-18526fc7-82bb-47f6-9cf8-c24f4ee247ce tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:46.857916 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bea3fab1-a680-4465-9509-5a73df01cc8d req-18526fc7-82bb-47f6-9cf8-c24f4ee247ce tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1026592 May 24 11:21:46.863263 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-684229'] for router, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:46.864160 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 684315b8-ba5e-4d23-8963-64aca3b067cb {{(pid=91820) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 11:21:46.882094 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] Notify callbacks [] for router, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:46.885642 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-15092ac8-1bcb-407f-8359-11ce687c7b96 admin admin] 10.222.0.18 "DELETE /v2.0/routers/684315b8-ba5e-4d23-8963-64aca3b067cb HTTP/1.1" status: 204 len: 168 time: 1.3983808 May 24 11:21:46.886491 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59086) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:46.923701 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bea3fab1-a680-4465-9509-5a73df01cc8d req-d28963f1-234e-42db-a50f-139eee7e824b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0576491 May 24 11:21:46.962258 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_v2 [None req-c1076fb4-381c-4a4f-9e8d-9f68158c7a2c admin admin] Deleting subnet 6a80c579-20d5-46c7-8c07-6438f1367afd {{(pid=91820) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:21:46.965768 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c1076fb4-381c-4a4f-9e8d-9f68158c7a2c admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:47.015578 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c1076fb4-381c-4a4f-9e8d-9f68158c7a2c admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534861947'] for subnet, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:47.135255 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c1076fb4-381c-4a4f-9e8d-9f68158c7a2c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:47.157969 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 90af9233-b13f-40e5-a0f8-872fe40ddfed no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:21:47.194369 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 90af9233-b13f-40e5-a0f8-872fe40ddfed {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:21:47.194731 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 90af9233-b13f-40e5-a0f8-872fe40ddfed not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:21:47.400082 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c1076fb4-381c-4a4f-9e8d-9f68158c7a2c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534788488', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535456750'] for subnet, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:47.407342 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b44e6310-1302-4363-ae7d-5eca7b23c4e3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:47.446847 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c1076fb4-381c-4a4f-9e8d-9f68158c7a2c admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:47.448086 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-03e5bc13-91ee-440d-87fb-1c8669a1d8ec None None] Network 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:21:47.449573 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c1076fb4-381c-4a4f-9e8d-9f68158c7a2c admin admin] Pushing event deleted for resources: {'Subnet': ['ID=6a80c579-20d5-46c7-8c07-6438f1367afd,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:47.451850 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-c1076fb4-381c-4a4f-9e8d-9f68158c7a2c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:47.458428 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-c1076fb4-381c-4a4f-9e8d-9f68158c7a2c admin admin] 10.222.0.18 "DELETE /v2.0/subnets/6a80c579-20d5-46c7-8c07-6438f1367afd HTTP/1.1" status: 204 len: 168 time: 0.5703347 May 24 11:21:47.461208 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59090) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:47.675432 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:47.712664 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-90bd5ee8-51c8-4080-b0fe-668db0e19ac8 None None] DHCP port deletion for 0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 request from dv-cinder-784092-9-iscsi {{(pid=91821) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 11:21:47.750152 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Deleting port f5a705e5-e33a-4169-b3ca-2428f9726466 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:21:47.750499 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:47.815534 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-90bd5ee8-51c8-4080-b0fe-668db0e19ac8 None None] Deleting port f5a705e5-e33a-4169-b3ca-2428f9726466 {{(pid=91821) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:21:47.815910 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-90bd5ee8-51c8-4080-b0fe-668db0e19ac8 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:47.861814 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] For port f5a705e5-e33a-4169-b3ca-2428f9726466, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=f5a705e5-e33a-4169-b3ca-2428f9726466,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:47.862231 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:47.919486 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-90bd5ee8-51c8-4080-b0fe-668db0e19ac8 None None] For port f5a705e5-e33a-4169-b3ca-2428f9726466, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=f5a705e5-e33a-4169-b3ca-2428f9726466,segment=NetworkSegment(666aa9e5-b138-4841-9cc4-ecbe52ac33c1),segment_id=666aa9e5-b138-4841-9cc4-ecbe52ac33c1)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:47.919825 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-90bd5ee8-51c8-4080-b0fe-668db0e19ac8 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:47.928154 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 9376988c-0749-43f3-a506-c8c29a15cbcf details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:21:48.012949 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Calling delete_port for f5a705e5-e33a-4169-b3ca-2428f9726466 owned by network:dhcp {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:21:48.084753 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-90bd5ee8-51c8-4080-b0fe-668db0e19ac8 None None] Calling delete_port for f5a705e5-e33a-4169-b3ca-2428f9726466 owned by network:dhcp {{(pid=91821) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:21:48.085032 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:48.092286 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1fd86000-c7fa-4260-9cae-1d55347007fe service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:48.097221 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.db.resource_extend [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] It took 0.11 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 24 11:21:48.154925 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Pushing event deleted for resources: {'Port': ['ID=f5a705e5-e33a-4169-b3ca-2428f9726466,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:48.157082 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] No DVR routers for this DVR port f5a705e5-e33a-4169-b3ca-2428f9726466 on host dv-cinder-784092-9-iscsi {{(pid=91820) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:21:48.158825 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:48.196754 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1b7fde24-d90e-45f4-b776-d74c700c7bce req-410c372e-b30c-42c4-91a0-97452c66f4ca tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:48.205897 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1b7fde24-d90e-45f4-b776-d74c700c7bce req-410c372e-b30c-42c4-91a0-97452c66f4ca tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1107187 May 24 11:21:48.276773 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1b7fde24-d90e-45f4-b776-d74c700c7bce req-fbd597ca-04fd-4e78-af16-53527de17088 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0672667 May 24 11:21:48.300535 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:48.317237 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397']}} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:48.330780 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '666aa9e5-b138-4841-9cc4-ecbe52ac33c1') {'for_net_delete': True} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:48.331110 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '666aa9e5-b138-4841-9cc4-ecbe52ac33c1') {} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:48.344654 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534861936'] for segment, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:48.363016 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:48.373451 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Releasing vlan 539 on physical network physnet1 to pool {{(pid=91820) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:21:48.373809 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535262766'] for segment, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: Traceback (most recent call last): May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: timer() May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: cb(*args, **kw) May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: return func(*args, **kwargs) May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: self._notify() May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: self.callback(batched_events) May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: event.method(event) May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: return f(self, *a, **k) May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._get(url).json() May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: return self.request(url, 'GET', **kwargs) May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:21:48.388166 dv-cinder-784092-9-iscsi neutron-server[91820]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:21:48.451828 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=9376988c-0749-43f3-a506-c8c29a15cbcf,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:48.478333 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'port_id': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'mac_address': 'fa:16:3e:aa:5b:b7', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 500, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:21:48.482761 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-90bd5ee8-51c8-4080-b0fe-668db0e19ac8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093'] for port, after_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:48.488023 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-1ff26346-8675-43df-b264-74dc1c913b76 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:48.491556 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:48.503701 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-11cf7aff-47ff-48d5-8bcf-6bb84c880646 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:48.537388 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:48.555911 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Pushing event deleted for resources: {'Network': ['ID=0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:48.562725 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:48.574594 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-90bd5ee8-51c8-4080-b0fe-668db0e19ac8 None None] Pushing event deleted for resources: {'Port': ['ID=f5a705e5-e33a-4169-b3ca-2428f9726466,revision_number=None']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:48.576535 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59120) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:48.577660 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-90bd5ee8-51c8-4080-b0fe-668db0e19ac8 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:48.578039 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-30b00c27-c829-4c11-a17e-30f0c4bac015 admin admin] 10.222.0.18 "DELETE /v2.0/networks/0ed4b4d6-5369-4cbf-b1e9-d3eb53cfa397 HTTP/1.1" status: 204 len: 168 time: 1.1144567 May 24 11:21:48.578831 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 54c19980-7452-40ac-b0f5-c89b800a3d24 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:21:48.619890 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-90bd5ee8-51c8-4080-b0fe-668db0e19ac8 None None] No DVR routers for this DVR port f5a705e5-e33a-4169-b3ca-2428f9726466 on host dv-cinder-784092-9-iscsi {{(pid=91821) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:21:48.637439 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 54c19980-7452-40ac-b0f5-c89b800a3d24 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:21:48.638132 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 54c19980-7452-40ac-b0f5-c89b800a3d24 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:21:48.638744 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device f5a705e5-e33a-4169-b3ca-2428f9726466 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:21:48.685870 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: f5a705e5-e33a-4169-b3ca-2428f9726466 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:21:48.686222 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device f5a705e5-e33a-4169-b3ca-2428f9726466 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:21:48.700522 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:48.731520 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: ACTIVE current_port_status: BUILD port_id 9376988c-0749-43f3-a506-c8c29a15cbcf {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:21:48.760569 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=9376988c-0749-43f3-a506-c8c29a15cbcf,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:48.764734 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:48.766927 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:48.845714 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=9376988c-0749-43f3-a506-c8c29a15cbcf,revision_number=6']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:48.849073 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:48.857950 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['9376988c-0749-43f3-a506-c8c29a15cbcf'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:21:49.300873 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-687465'] for router_interface, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:49.767674 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-190d5772-ae04-4f82-a041-010ea10b6e1e req-397834b4-446e-432b-b3d4-cd147c3c332e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:49.771653 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-190d5772-ae04-4f82-a041-010ea10b6e1e req-397834b4-446e-432b-b3d4-cd147c3c332e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 945 time: 0.2636967 May 24 11:21:49.945651 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 9376988c-0749-43f3-a506-c8c29a15cbcf up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:21:49.982820 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-190d5772-ae04-4f82-a041-010ea10b6e1e req-92072395-9667-45bc-a1f3-3c7d6a33b08a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1369746 May 24 11:21:50.064975 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 9376988c-0749-43f3-a506-c8c29a15cbcf triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:21:50.065664 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:50.158162 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] Deleting port 1a4dddfd-3c51-421f-b942-92a179e987b8 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:21:50.158507 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:50.217751 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:50.248690 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:21:50.258805 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=9376988c-0749-43f3-a506-c8c29a15cbcf,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:50.273720 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:50.276007 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:50.316145 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] For port 1a4dddfd-3c51-421f-b942-92a179e987b8, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=1a4dddfd-3c51-421f-b942-92a179e987b8,segment=NetworkSegment(88ac69f8-b455-4003-825b-2df6e05e49fa),segment_id=88ac69f8-b455-4003-825b-2df6e05e49fa)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:50.316652 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:50.422837 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=9376988c-0749-43f3-a506-c8c29a15cbcf,revision_number=7']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:50.431406 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.155s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:50.457641 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'code': 200} May 24 11:21:50.473702 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] Calling delete_port for 1a4dddfd-3c51-421f-b942-92a179e987b8 owned by network:router_interface {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:21:50.829827 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:50.833829 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-57089403-eb80-4596-a6a6-0f2172142a46 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:50.890377 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] Pushing event deleted for resources: {'Port': ['ID=1a4dddfd-3c51-421f-b942-92a179e987b8,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:50.895829 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:51.395786 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-537767', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-685561'] for router_interface, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:51.416856 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17c4438d-d0ea-4442-a9e3-dd1120c6c4b3 req-aca55537-5281-4156-9b1f-44933a0dec20 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:51.418336 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-17c4438d-d0ea-4442-a9e3-dd1120c6c4b3 req-aca55537-5281-4156-9b1f-44933a0dec20 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.2783611 May 24 11:21:51.599831 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-17c4438d-d0ea-4442-a9e3-dd1120c6c4b3 req-a230d794-4eb4-4b6a-9dcc-a60bf17caadc tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1299336 May 24 11:21:51.843799 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] Router d37dc0a7-5e53-4516-8020-b2990086e8c5 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:21:52.029552 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:52.101190 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:21:52.120263 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] 10.222.0.18 "PUT /v2.0/routers/d37dc0a7-5e53-4516-8020-b2990086e8c5/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.5394711 May 24 11:21:52.120585 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59142) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:52.129331 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d37a938a-3f0d-4341-96ff-c8cf065d79e1 None None] Sync routers for ids ['d37dc0a7-5e53-4516-8020-b2990086e8c5'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:21:52.153430 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d37a938a-3f0d-4341-96ff-c8cf065d79e1 None None] Routers {'d37dc0a7-5e53-4516-8020-b2990086e8c5'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:21:52.155913 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-d37a938a-3f0d-4341-96ff-c8cf065d79e1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:21:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['d37dc0a7-5e53-4516-8020-b2990086e8c5'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:52.313059 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:21:52.373498 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Notify callbacks [] for router, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:52.382141 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-d37a938a-3f0d-4341-96ff-c8cf065d79e1 None None] Set router d37dc0a7-5e53-4516-8020-b2990086e8c5 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:21:52.748910 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:52.878619 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-d37a938a-3f0d-4341-96ff-c8cf065d79e1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'name': 'tempest-TestMinimumBasicScenario-648060998-router', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181'}], 'enable_snat': True}, 'gw_port_id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:44Z', 'updated_at': '2021-05-24T11:21:50Z', 'revision_number': 5, 'project_id': '675d4e36e7c14b46864dd445f05bc0ec', 'gw_port': {'id': '23bbb434-e051-4fbc-8e80-f213cd1ab793', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:02:04:4a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 385, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.181', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:46Z', 'updated_at': '2021-05-24T11:18:59Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:52.884285 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d37a938a-3f0d-4341-96ff-c8cf065d79e1 None None] Checking router: d37dc0a7-5e53-4516-8020-b2990086e8c5 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:21:52.891113 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Deleting port 23bbb434-e051-4fbc-8e80-f213cd1ab793 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:21:52.891367 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:52.895994 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d37a938a-3f0d-4341-96ff-c8cf065d79e1 None None] Routers {'d37dc0a7-5e53-4516-8020-b2990086e8c5'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:21:52.896917 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-d37a938a-3f0d-4341-96ff-c8cf065d79e1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:21:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['d37dc0a7-5e53-4516-8020-b2990086e8c5'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:52.935706 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bb9e8399-dbea-4b6c-b979-bc194eaa6887 req-9f15f61b-b8a7-4ebb-a296-ae41fe3153c6 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:52.946131 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bb9e8399-dbea-4b6c-b979-bc194eaa6887 req-9f15f61b-b8a7-4ebb-a296-ae41fe3153c6 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1850750 May 24 11:21:53.008075 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bb9e8399-dbea-4b6c-b979-bc194eaa6887 req-a4369b40-f11f-4035-8995-5252e7d2a363 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0605862 May 24 11:21:53.096694 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] For port 23bbb434-e051-4fbc-8e80-f213cd1ab793, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=23bbb434-e051-4fbc-8e80-f213cd1ab793,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:53.097265 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:53.123227 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-d37a938a-3f0d-4341-96ff-c8cf065d79e1 None None] Set router d37dc0a7-5e53-4516-8020-b2990086e8c5 gateway port host: None {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:21:53.330392 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-d37a938a-3f0d-4341-96ff-c8cf065d79e1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'name': 'tempest-TestMinimumBasicScenario-648060998-router', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:44Z', 'updated_at': '2021-05-24T11:21:52Z', 'revision_number': 6, 'project_id': '675d4e36e7c14b46864dd445f05bc0ec', 'gw_port_host': None}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:53.341167 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d37a938a-3f0d-4341-96ff-c8cf065d79e1 None None] The sync data for ids ['d37dc0a7-5e53-4516-8020-b2990086e8c5'] in dv-cinder-784092-9-iscsi is: [{'id': 'd37dc0a7-5e53-4516-8020-b2990086e8c5', 'name': 'tempest-TestMinimumBasicScenario-648060998-router', 'tenant_id': '675d4e36e7c14b46864dd445f05bc0ec', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:44Z', 'updated_at': '2021-05-24T11:21:52Z', 'revision_number': 6, 'project_id': '675d4e36e7c14b46864dd445f05bc0ec', 'gw_port_host': None}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:21:53.376068 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Calling delete_port for 23bbb434-e051-4fbc-8e80-f213cd1ab793 owned by network:router_gateway {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:21:53.432413 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3403d3b6-ec5a-45f2-8051-8dacb867d66d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:53.584320 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:53.589148 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b584ca32-cbdb-469c-ac65-f76386ab12c3 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:53.636919 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Pushing event deleted for resources: {'Port': ['ID=23bbb434-e051-4fbc-8e80-f213cd1ab793,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:53.639576 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:53.744769 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-686964'] for router_gateway, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:53.936707 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-684537', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4171'] for router, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:53.941553 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:53.970041 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-684229'] for router, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:53.970688 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Fanout notify agent at l3_agent the message router_deleted on router d37dc0a7-5e53-4516-8020-b2990086e8c5 {{(pid=91820) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 11:21:53.983732 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] Notify callbacks [] for router, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:53.987415 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-d2b31715-a46b-4e82-9b35-a46bc3177cc6 admin admin] 10.222.0.18 "DELETE /v2.0/routers/d37dc0a7-5e53-4516-8020-b2990086e8c5 HTTP/1.1" status: 204 len: 168 time: 1.8633671 May 24 11:21:53.987994 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59152) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:54.086547 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_v2 [None req-697ffd6c-5994-47bc-871a-16649c319599 admin admin] Deleting subnet a90ff9ef-eadb-4731-b071-d0c9fe13cb00 {{(pid=91820) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:21:54.090099 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-697ffd6c-5994-47bc-871a-16649c319599 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:54.148911 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-697ffd6c-5994-47bc-871a-16649c319599 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534861947'] for subnet, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:54.250160 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f5f01bfa-d7a6-4837-a85f-b9a7c0d42d91 req-d231d0d4-2984-48c2-8b99-789f9ce2cf62 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:54.251642 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f5f01bfa-d7a6-4837-a85f-b9a7c0d42d91 req-d231d0d4-2984-48c2-8b99-789f9ce2cf62 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1163604 May 24 11:21:54.279353 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-697ffd6c-5994-47bc-871a-16649c319599 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:54.313144 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f5f01bfa-d7a6-4837-a85f-b9a7c0d42d91 req-8c271471-e361-4842-a76b-f4096d01dd7d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0516114 May 24 11:21:54.539817 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 23bbb434-e051-4fbc-8e80-f213cd1ab793 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:21:54.594831 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 23bbb434-e051-4fbc-8e80-f213cd1ab793 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:21:54.595288 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 23bbb434-e051-4fbc-8e80-f213cd1ab793 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:21:54.595791 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 1a4dddfd-3c51-421f-b942-92a179e987b8 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:21:54.651447 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 1a4dddfd-3c51-421f-b942-92a179e987b8 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:21:54.651909 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 1a4dddfd-3c51-421f-b942-92a179e987b8 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:21:54.665346 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-697ffd6c-5994-47bc-871a-16649c319599 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534788488', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535456750'] for subnet, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:54.671404 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-2241922d-96fe-42d8-9fe4-ef956b21ae8f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:54.723929 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-697ffd6c-5994-47bc-871a-16649c319599 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=a90ff9ef-eadb-4731-b071-d0c9fe13cb00,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:54.745197 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-b183dfd0-e5e5-48a0-861b-d1f6daaa0dce None None] Network 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:21:54.749973 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-697ffd6c-5994-47bc-871a-16649c319599 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:54.751160 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-697ffd6c-5994-47bc-871a-16649c319599 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:54.762165 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-697ffd6c-5994-47bc-871a-16649c319599 admin admin] 10.222.0.18 "DELETE /v2.0/subnets/a90ff9ef-eadb-4731-b071-d0c9fe13cb00 HTTP/1.1" status: 204 len: 168 time: 0.7689486 May 24 11:21:54.763356 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59158) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:54.889439 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:55.048745 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Deleting port 3b438696-d4a6-404d-a9c9-be9bffe037af {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:21:55.049211 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:55.210054 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] For port 3b438696-d4a6-404d-a9c9-be9bffe037af, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=3b438696-d4a6-404d-a9c9-be9bffe037af,segment=NetworkSegment(88ac69f8-b455-4003-825b-2df6e05e49fa),segment_id=88ac69f8-b455-4003-825b-2df6e05e49fa)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:55.210557 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:55.272627 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-2fd12ac0-4083-45e8-a4ca-ee56694e5016 None None] DHCP port deletion for 9672fa3f-743d-4d68-8dd2-0bf684ca5e6a request from dv-cinder-784092-9-iscsi {{(pid=91821) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 11:21:55.464499 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Calling delete_port for 3b438696-d4a6-404d-a9c9-be9bffe037af owned by network:dhcp {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:21:55.520888 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-2fd12ac0-4083-45e8-a4ca-ee56694e5016 None None] Deleting port 3b438696-d4a6-404d-a9c9-be9bffe037af {{(pid=91821) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:21:55.521628 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-2fd12ac0-4083-45e8-a4ca-ee56694e5016 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:55.662456 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:55.681696 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-1ca83cc2-2182-4c6c-a353-9911e3c8771f tempest-VolumesActionsTest-907817959 tempest-VolumesActionsTest-907817959-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:55.751605 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-2fd12ac0-4083-45e8-a4ca-ee56694e5016 None None] For port 3b438696-d4a6-404d-a9c9-be9bffe037af, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=3b438696-d4a6-404d-a9c9-be9bffe037af,segment=NetworkSegment(88ac69f8-b455-4003-825b-2df6e05e49fa),segment_id=88ac69f8-b455-4003-825b-2df6e05e49fa)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:21:55.753241 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-2fd12ac0-4083-45e8-a4ca-ee56694e5016 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:55.903403 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Pushing event deleted for resources: {'Port': ['ID=3b438696-d4a6-404d-a9c9-be9bffe037af,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:55.905559 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] No DVR routers for this DVR port 3b438696-d4a6-404d-a9c9-be9bffe037af on host dv-cinder-784092-9-iscsi {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:21:55.954390 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.273s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:55.971448 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-2fd12ac0-4083-45e8-a4ca-ee56694e5016 None None] Calling delete_port for 3b438696-d4a6-404d-a9c9-be9bffe037af owned by network:dhcp {{(pid=91821) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:21:55.994248 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-9854a42a-3cb6-4b2d-9a05-4cd3b4c650ba None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:56.009153 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:56.054656 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-2fd12ac0-4083-45e8-a4ca-ee56694e5016 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093'] for port, after_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:56.060810 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:56.095489 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:56.111183 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-22a5e3e6-6fcc-4400-85c7-62fea41531b1 req-141b75e1-1037-4591-81c0-23325b386420 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:56.112766 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-22a5e3e6-6fcc-4400-85c7-62fea41531b1 req-141b75e1-1037-4591-81c0-23325b386420 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.6243539 May 24 11:21:56.119813 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['9672fa3f-743d-4d68-8dd2-0bf684ca5e6a']}} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:56.150090 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2fd12ac0-4083-45e8-a4ca-ee56694e5016 None None] No DVR routers for this DVR port 3b438696-d4a6-404d-a9c9-be9bffe037af on host dv-cinder-784092-9-iscsi {{(pid=91821) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:21:56.150635 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '88ac69f8-b455-4003-825b-2df6e05e49fa') {'for_net_delete': True} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:56.150938 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '88ac69f8-b455-4003-825b-2df6e05e49fa') {} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:21:56.165352 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2fd12ac0-4083-45e8-a4ca-ee56694e5016 None None] Pushing event deleted for resources: {'Port': ['ID=3b438696-d4a6-404d-a9c9-be9bffe037af,revision_number=None']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:56.167158 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-2fd12ac0-4083-45e8-a4ca-ee56694e5016 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:56.169899 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534853183'] for segment, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:56.230445 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:56.256116 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-22a5e3e6-6fcc-4400-85c7-62fea41531b1 req-16e2a147-eacd-434e-b03a-86e658ab945e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1285386 May 24 11:21:56.269563 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Releasing vlan 531 on physical network physnet1 to pool {{(pid=91819) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:21:56.270621 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535155490'] for segment, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: Traceback (most recent call last): May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: timer() May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: cb(*args, **kw) May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: return func(*args, **kwargs) May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: self._notify() May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: self.callback(batched_events) May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: event.method(event) May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: return f(self, *a, **k) May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._get(url).json() May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: return self.request(url, 'GET', **kwargs) May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:21:56.303829 dv-cinder-784092-9-iscsi neutron-server[91819]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:21:56.442178 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:56.454459 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f4994c91-cfcd-4c54-8dc8-807773ee0605 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:56.472304 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:56.497673 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] 10.222.0.18 "DELETE /v2.0/networks/9672fa3f-743d-4d68-8dd2-0bf684ca5e6a HTTP/1.1" status: 204 len: 168 time: 1.7307091 May 24 11:21:56.498432 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Pushing event deleted for resources: {'Network': ['ID=9672fa3f-743d-4d68-8dd2-0bf684ca5e6a,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:56.501011 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:56.536930 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 3b438696-d4a6-404d-a9c9-be9bffe037af no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:21:56.609210 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 3b438696-d4a6-404d-a9c9-be9bffe037af {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:21:56.609471 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 3b438696-d4a6-404d-a9c9-be9bffe037af not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:21:56.791172 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59174) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:56.880271 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-423f1b1c-cab0-477a-8586-3206cadde9eb admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=788050c389224d17b163f24bb67902bf&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0875390 May 24 11:21:56.884468 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59178) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:56.982891 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-84090ff6-64f3-4a36-8d2b-ad07c435a21b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:56.999170 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:57.021547 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-84090ff6-64f3-4a36-8d2b-ad07c435a21b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:57.112306 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-84090ff6-64f3-4a36-8d2b-ad07c435a21b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:57.130967 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e8d1acb4-9a4a-460e-877d-f32f4074f12d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:57.143841 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-84090ff6-64f3-4a36-8d2b-ad07c435a21b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=29c10cc8-55e7-4791-971b-485a28ba237e,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:57.148249 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-84090ff6-64f3-4a36-8d2b-ad07c435a21b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:57.159059 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-84090ff6-64f3-4a36-8d2b-ad07c435a21b admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:57.164344 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-84090ff6-64f3-4a36-8d2b-ad07c435a21b admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/29c10cc8-55e7-4791-971b-485a28ba237e HTTP/1.1" status: 204 len: 168 time: 0.2787936 May 24 11:21:57.592427 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3077a303-1f79-44bd-a3b7-b69a9992f402 req-a2e220db-7377-463f-8bf4-10f6ecae9563 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:57.595222 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3077a303-1f79-44bd-a3b7-b69a9992f402 req-a2e220db-7377-463f-8bf4-10f6ecae9563 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1604261 May 24 11:21:57.677606 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3077a303-1f79-44bd-a3b7-b69a9992f402 req-3f8b6a1c-79a6-46ae-9976-887fe1ae11ac tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0746965 May 24 11:21:57.932093 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59190) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:58.000520 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-4b31d87b-2238-4310-a0d1-d28119403a45 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=675d4e36e7c14b46864dd445f05bc0ec&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0666330 May 24 11:21:58.004695 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59194) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:21:58.105917 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e023b9ef-bb7a-43cb-82f5-cc4cc363c8d7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:58.141901 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e023b9ef-bb7a-43cb-82f5-cc4cc363c8d7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:58.229336 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e023b9ef-bb7a-43cb-82f5-cc4cc363c8d7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:58.233603 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:21:58.248166 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e023b9ef-bb7a-43cb-82f5-cc4cc363c8d7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b1c18bf3-361f-4b94-a982-45ecca0f10fd,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:21:58.251301 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e023b9ef-bb7a-43cb-82f5-cc4cc363c8d7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:21:58.253382 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e023b9ef-bb7a-43cb-82f5-cc4cc363c8d7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:58.257373 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-e023b9ef-bb7a-43cb-82f5-cc4cc363c8d7 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/b1c18bf3-361f-4b94-a982-45ecca0f10fd HTTP/1.1" status: 204 len: 168 time: 0.2487257 May 24 11:21:58.955180 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:21:58.964826 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cbea7a33-840e-4d0b-8eb3-c95a92a0d9a6 req-c7388f36-6487-4d3c-98ac-b7b5d6b3b1ba tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:58.967510 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cbea7a33-840e-4d0b-8eb3-c95a92a0d9a6 req-c7388f36-6487-4d3c-98ac-b7b5d6b3b1ba tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1153662 May 24 11:21:59.020689 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cbea7a33-840e-4d0b-8eb3-c95a92a0d9a6 req-8247169c-accf-47c5-9e63-38c0322210ff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0439761 May 24 11:21:59.842582 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d77c8ba-46bd-46f6-b511-59f54524aa29 req-ddc2a99e-7c40-4cbe-8266-601feb5e4dc2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:59.844041 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d77c8ba-46bd-46f6-b511-59f54524aa29 req-ddc2a99e-7c40-4cbe-8266-601feb5e4dc2 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 1099 time: 0.0779102 May 24 11:21:59.979508 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d77c8ba-46bd-46f6-b511-59f54524aa29 req-f56f715e-403f-4627-86a8-722bdcb73b1d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:21:59.980992 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d77c8ba-46bd-46f6-b511-59f54524aa29 req-f56f715e-403f-4627-86a8-722bdcb73b1d service neutron] 10.222.0.93 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 917 time: 0.1307297 May 24 11:22:00.086758 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d77c8ba-46bd-46f6-b511-59f54524aa29 req-3aa84c29-da11-4b06-8264-bd05d9573dcc service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=9376988c-0749-43f3-a506-c8c29a15cbcf HTTP/1.1" status: 200 len: 1001 time: 0.0995619 May 24 11:22:00.196324 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d77c8ba-46bd-46f6-b511-59f54524aa29 req-7e51c9fb-e830-451b-82e7-00aad5581869 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:00.197871 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d77c8ba-46bd-46f6-b511-59f54524aa29 req-7e51c9fb-e830-451b-82e7-00aad5581869 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=12ff35e8-9af8-4339-95c9-c62c90d8c6fd HTTP/1.1" status: 200 len: 855 time: 0.1041567 May 24 11:22:00.302162 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aca1fe52-c3de-4481-8715-b79a766337dd req-4b7d87bc-a4df-4ae9-a0ce-901a28fddbcb tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:00.303649 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-aca1fe52-c3de-4481-8715-b79a766337dd req-4b7d87bc-a4df-4ae9-a0ce-901a28fddbcb tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.1582530 May 24 11:22:00.331047 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d77c8ba-46bd-46f6-b511-59f54524aa29 req-76672d55-c865-49d6-8a1f-565d203f79fa service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:00.332404 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d77c8ba-46bd-46f6-b511-59f54524aa29 req-76672d55-c865-49d6-8a1f-565d203f79fa service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=7877e643-fe9e-4041-91fa-3a4f44279fd2&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1259067 May 24 11:22:00.424662 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-aca1fe52-c3de-4481-8715-b79a766337dd req-ddc1f2e1-4214-413e-b516-fce6f8636352 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1152768 May 24 11:22:00.493158 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d77c8ba-46bd-46f6-b511-59f54524aa29 req-d30e27bc-a542-40c6-9814-9853a4e01dcf service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1490724 May 24 11:22:00.626066 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d77c8ba-46bd-46f6-b511-59f54524aa29 req-eab95491-6b49-4a45-aca8-1654f5b43b2c service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1256194 May 24 11:22:01.630108 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd3c66ae-b048-4b41-a048-14ffcb1c374d req-f37943d5-4924-4e98-814f-35ed1a990449 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:01.632137 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fd3c66ae-b048-4b41-a048-14ffcb1c374d req-f37943d5-4924-4e98-814f-35ed1a990449 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.0913141 May 24 11:22:01.678279 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fd3c66ae-b048-4b41-a048-14ffcb1c374d req-ad3e642a-d0ee-41a0-92a3-578c3f5126c5 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0403636 May 24 11:22:02.870564 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a04c59e3-1b26-4d50-8e17-35959bbb6e06 req-cc9dbf0b-954e-42b3-80ec-a474eca11873 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:02.872041 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a04c59e3-1b26-4d50-8e17-35959bbb6e06 req-cc9dbf0b-954e-42b3-80ec-a474eca11873 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.0808032 May 24 11:22:02.918744 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a04c59e3-1b26-4d50-8e17-35959bbb6e06 req-d6bdba58-feb2-4a9a-8a7a-fcc9e05ddf48 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0405040 May 24 11:22:03.876223 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dee9955f-4be1-44d2-94ee-7756c0e1106d req-63cba6b3-e5d7-4c27-b79c-7012d1054f98 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:03.876223 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-dee9955f-4be1-44d2-94ee-7756c0e1106d req-63cba6b3-e5d7-4c27-b79c-7012d1054f98 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 946 time: 0.0927649 May 24 11:22:03.930331 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-dee9955f-4be1-44d2-94ee-7756c0e1106d req-2c51628c-822b-43d3-bef4-94eba5697a7d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0450919 May 24 11:22:03.999868 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 9376988c-0749-43f3-a506-c8c29a15cbcf no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:22:04.123772 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:04.162314 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:22:04.172508 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=9376988c-0749-43f3-a506-c8c29a15cbcf,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:22:04.179842 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:04.183518 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:04.215992 dv-cinder-784092-9-iscsi neutron-server[91821]: WARNING neutron.notifiers.nova [-] Nova event: {'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'name': 'network-vif-unplugged', 'status': 'failed', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'code': 422} returned with failed status May 24 11:22:04.238189 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59270) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:22:04.260943 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=9376988c-0749-43f3-a506-c8c29a15cbcf,revision_number=8']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:04.264967 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:04.360179 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-1b9b4ffe-4d30-4795-b4af-da62606cdc29 service neutron] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.1203074 May 24 11:22:05.234113 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd2e4022-29c8-4870-b0a8-8fa31ed9a0b2 req-d9d39bfa-4507-4d65-8e19-d008909b9b31 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:05.235750 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cd2e4022-29c8-4870-b0a8-8fa31ed9a0b2 req-d9d39bfa-4507-4d65-8e19-d008909b9b31 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.1901584 May 24 11:22:05.280985 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cd2e4022-29c8-4870-b0a8-8fa31ed9a0b2 req-6c08fd21-60c0-4dce-9384-16396413c776 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0389872 May 24 11:22:05.990502 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-07329da4-9380-4992-9d91-fcfbff088d18 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:05.992041 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-07329da4-9380-4992-9d91-fcfbff088d18 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f&tenant_id=05ec02ff35d2402e8d7bf064752e67ea HTTP/1.1" status: 200 len: 1097 time: 0.0719543 May 24 11:22:06.093901 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-923ba47d-0248-47b0-9a74-d77a45af2941 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:06.095367 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-923ba47d-0248-47b0-9a74-d77a45af2941 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 1097 time: 0.0772252 May 24 11:22:06.230193 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-0d4092d4-3866-4996-9658-9cf826833687 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:06.231627 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-0d4092d4-3866-4996-9658-9cf826833687 service neutron] 10.222.0.93 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 917 time: 0.1286452 May 24 11:22:06.337562 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-bc281961-be2e-4949-ba32-e6001a63efa4 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=9376988c-0749-43f3-a506-c8c29a15cbcf HTTP/1.1" status: 200 len: 999 time: 0.0974767 May 24 11:22:06.434691 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-7c72174c-6445-45aa-ba31-cf09da95cf6d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:06.436116 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-7c72174c-6445-45aa-ba31-cf09da95cf6d service neutron] 10.222.0.93 "GET /v2.0/subnets?id=12ff35e8-9af8-4339-95c9-c62c90d8c6fd HTTP/1.1" status: 200 len: 855 time: 0.0925224 May 24 11:22:06.544056 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f4c980a-8457-44da-8493-b9279ac33142 req-7dee0bff-87d4-48df-b087-140ccff4d656 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:06.546852 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6f4c980a-8457-44da-8493-b9279ac33142 req-7dee0bff-87d4-48df-b087-140ccff4d656 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.1514113 May 24 11:22:06.555620 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-187b0ab5-0ff6-4580-800c-fd7c0026c2b0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:06.557131 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-187b0ab5-0ff6-4580-800c-fd7c0026c2b0 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=7877e643-fe9e-4041-91fa-3a4f44279fd2&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1136296 May 24 11:22:06.681183 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6f4c980a-8457-44da-8493-b9279ac33142 req-3650963c-3b73-4edc-a921-6348e1546d69 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1230795 May 24 11:22:06.721872 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-d4235fad-ee0b-489e-af72-afe291fcb569 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1491635 May 24 11:22:06.853568 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-92f1efb0-306e-4639-99df-7d67f7576ebe req-d08ad8b9-0ab4-4555-9e4b-1f30443ae7f1 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1251795 May 24 11:22:07.899787 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-355c0449-546c-4e2c-8c57-c62268435aa4 req-e038e7ef-731e-4411-8e7c-0a37ce02f18a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:07.901460 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-355c0449-546c-4e2c-8c57-c62268435aa4 req-e038e7ef-731e-4411-8e7c-0a37ce02f18a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0936143 May 24 11:22:07.952054 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-355c0449-546c-4e2c-8c57-c62268435aa4 req-888d42fa-5959-4316-b4ef-9fddcd570713 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0448532 May 24 11:22:09.158708 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-23b39b4f-e7b6-4a63-89fa-2c6cd754c37c req-b6e9f0f2-5f95-48b0-8288-c08bf50992d0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:09.165683 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-23b39b4f-e7b6-4a63-89fa-2c6cd754c37c req-b6e9f0f2-5f95-48b0-8288-c08bf50992d0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0903451 May 24 11:22:09.208421 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-23b39b4f-e7b6-4a63-89fa-2c6cd754c37c req-c911e5e0-d440-485b-a171-d742c1c2e84d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0415983 May 24 11:22:10.407607 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-335110c4-a3b8-45e1-96f2-74f52f70782b req-e686b22f-dd29-410f-9960-65b1883dd65a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:10.409040 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-335110c4-a3b8-45e1-96f2-74f52f70782b req-e686b22f-dd29-410f-9960-65b1883dd65a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0861051 May 24 11:22:10.463608 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-335110c4-a3b8-45e1-96f2-74f52f70782b req-53f621c4-c28c-4dc4-a242-eb0b829e79f4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0488682 May 24 11:22:11.660257 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b26e949-afed-4ebf-81d4-fe86da274a2e req-f5a512d3-5c7f-4938-9033-f28231308bf0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:11.661773 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b26e949-afed-4ebf-81d4-fe86da274a2e req-f5a512d3-5c7f-4938-9033-f28231308bf0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0830374 May 24 11:22:11.713183 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b26e949-afed-4ebf-81d4-fe86da274a2e req-46767f72-0388-4696-b478-70c66bec3d69 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0455334 May 24 11:22:12.905585 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9b38cba-b4e2-453f-bdc9-b44f47cd5e2f req-147c5975-21e3-4a59-aca3-da9f273bd0e5 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:12.907036 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e9b38cba-b4e2-453f-bdc9-b44f47cd5e2f req-147c5975-21e3-4a59-aca3-da9f273bd0e5 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0820136 May 24 11:22:12.955142 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e9b38cba-b4e2-453f-bdc9-b44f47cd5e2f req-3c8addca-02b9-4c72-b71f-ca99376a7817 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0424986 May 24 11:22:14.164061 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8cd36900-89f3-41fe-a83b-6b6b4851c267 req-f43a5001-9bee-43cf-8c80-1c53bf442fd8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:14.165719 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8cd36900-89f3-41fe-a83b-6b6b4851c267 req-f43a5001-9bee-43cf-8c80-1c53bf442fd8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0896046 May 24 11:22:14.212806 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8cd36900-89f3-41fe-a83b-6b6b4851c267 req-401d112e-8f7e-43c6-9122-e68eac7dddc4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0414178 May 24 11:22:15.412904 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2620b1e3-b537-4490-9a48-e078f2cc0cec req-f6429281-8f9d-45ce-9445-ab232eda3506 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:15.414420 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2620b1e3-b537-4490-9a48-e078f2cc0cec req-f6429281-8f9d-45ce-9445-ab232eda3506 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0860856 May 24 11:22:15.460258 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2620b1e3-b537-4490-9a48-e078f2cc0cec req-974a5945-6f41-48a2-a9cf-aea69f569d5a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0400717 May 24 11:22:16.750586 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-12efb7b8-0460-4f24-a674-f9dc1366e557 req-4b2facbf-9017-4c1d-9064-c3c83a45d467 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:16.751943 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-12efb7b8-0460-4f24-a674-f9dc1366e557 req-4b2facbf-9017-4c1d-9064-c3c83a45d467 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.1760819 May 24 11:22:16.797412 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-12efb7b8-0460-4f24-a674-f9dc1366e557 req-ce79eb90-c1fd-4138-83b0-299adc7cf415 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0396540 May 24 11:22:17.993801 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6901ff69-2a28-4d1e-a505-49324c0d82f5 req-cec8ad5a-3ffb-4077-a86d-bb6ce584b4df tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:17.995251 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6901ff69-2a28-4d1e-a505-49324c0d82f5 req-cec8ad5a-3ffb-4077-a86d-bb6ce584b4df tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0814393 May 24 11:22:18.043322 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6901ff69-2a28-4d1e-a505-49324c0d82f5 req-7ab4b1a6-f8e5-4311-944f-13f4efbb5796 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0424426 May 24 11:22:18.968603 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-64735e34-5726-494c-9a58-fce528218c88 req-bde80ca8-636b-4e08-bd66-9fdd0eb96766 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0700309 May 24 11:22:19.057712 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-64735e34-5726-494c-9a58-fce528218c88 req-c4c9831b-a365-4f2c-b028-3ff2ad8bf376 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:19.059186 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-64735e34-5726-494c-9a58-fce528218c88 req-c4c9831b-a365-4f2c-b028-3ff2ad8bf376 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 1098 time: 0.0705171 May 24 11:22:19.206252 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-64735e34-5726-494c-9a58-fce528218c88 req-0f6f5065-a357-4b81-a62a-ef1dc869395f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:19.207684 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-64735e34-5726-494c-9a58-fce528218c88 req-0f6f5065-a357-4b81-a62a-ef1dc869395f service neutron] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 912 time: 0.1296380 May 24 11:22:19.241537 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca1878a5-c9b1-43bb-b9c1-a6dac15ed1cd req-406fc259-c5a3-4fc3-b75b-1f801c2bc466 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:19.242963 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ca1878a5-c9b1-43bb-b9c1-a6dac15ed1cd req-406fc259-c5a3-4fc3-b75b-1f801c2bc466 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0862753 May 24 11:22:19.288541 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ca1878a5-c9b1-43bb-b9c1-a6dac15ed1cd req-e92a4b79-ca86-431b-a2c2-f892ddc074b4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0398493 May 24 11:22:19.311741 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-64735e34-5726-494c-9a58-fce528218c88 req-fc03de83-83fc-40ff-ac4f-24a72e49198d service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=8a7c6451-6355-44ad-a36a-f02ffcfe252c HTTP/1.1" status: 200 len: 1001 time: 0.0979011 May 24 11:22:19.378017 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-64735e34-5726-494c-9a58-fce528218c88 req-3d9a1973-d7d9-4032-9088-3ed61d58d327 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:19.379427 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-64735e34-5726-494c-9a58-fce528218c88 req-3d9a1973-d7d9-4032-9088-3ed61d58d327 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.0611610 May 24 11:22:19.452937 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-64735e34-5726-494c-9a58-fce528218c88 req-c2bd4912-371f-44af-831b-f3f0ea406b78 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:19.454400 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-64735e34-5726-494c-9a58-fce528218c88 req-c2bd4912-371f-44af-831b-f3f0ea406b78 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0688438 May 24 11:22:19.587056 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-64735e34-5726-494c-9a58-fce528218c88 req-0ca74839-93d0-4618-9226-45ff7f82c853 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1260238 May 24 11:22:19.721037 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-64735e34-5726-494c-9a58-fce528218c88 req-4b51a584-1ca9-45b3-b97f-02ee8d415ddb service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1275105 May 24 11:22:20.478810 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1457e5e6-135d-4ab8-bca9-5a2fea6b9b17 req-31edb46a-6d05-4fb5-8f21-5e8e0cee5cfc tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:20.480239 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1457e5e6-135d-4ab8-bca9-5a2fea6b9b17 req-31edb46a-6d05-4fb5-8f21-5e8e0cee5cfc tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0805089 May 24 11:22:20.527768 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1457e5e6-135d-4ab8-bca9-5a2fea6b9b17 req-f683ab62-0169-4811-8148-fd204a41966f tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0419858 May 24 11:22:21.723444 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ac427896-ee8f-4234-adfe-33788222fdc5 req-4cad76f5-8c65-42f0-871e-bb48eaa2bca8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:21.724898 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ac427896-ee8f-4234-adfe-33788222fdc5 req-4cad76f5-8c65-42f0-871e-bb48eaa2bca8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0832648 May 24 11:22:21.768479 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ac427896-ee8f-4234-adfe-33788222fdc5 req-b61f4b11-1da9-46cb-a748-d25841c17995 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0380127 May 24 11:22:22.018764 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:22.963058 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6b235b6c-b8a6-40ce-9c0f-484d242bde5f req-d19016d5-4548-495a-8e8f-5a43f4c7b437 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:22.964547 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6b235b6c-b8a6-40ce-9c0f-484d242bde5f req-d19016d5-4548-495a-8e8f-5a43f4c7b437 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0812404 May 24 11:22:23.009457 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6b235b6c-b8a6-40ce-9c0f-484d242bde5f req-25886cbb-1ab7-4bba-8ef2-4947bf9f3c14 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0391724 May 24 11:22:23.417402 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-4e16e4e3-02d0-4dd4-81ec-b85d6f4c1a76 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:24.201050 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d33e1143-d904-4d6f-b873-1a75ba8ee5f4 req-3e2f002a-3171-4c62-b100-3795a70dd9f5 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:24.202522 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d33e1143-d904-4d6f-b873-1a75ba8ee5f4 req-3e2f002a-3171-4c62-b100-3795a70dd9f5 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.0795894 May 24 11:22:24.248166 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d33e1143-d904-4d6f-b873-1a75ba8ee5f4 req-0a2cb84d-0c6d-4453-9ddd-5441ad3af66a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0395343 May 24 11:22:25.529879 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-69adf4ac-2ba3-492f-aa7d-004b8a5f1db1 req-79aec374-3c46-4afc-8443-5fa174eaba85 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:25.531342 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-69adf4ac-2ba3-492f-aa7d-004b8a5f1db1 req-79aec374-3c46-4afc-8443-5fa174eaba85 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 944 time: 0.1739678 May 24 11:22:25.577364 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-69adf4ac-2ba3-492f-aa7d-004b8a5f1db1 req-f22a8b58-fd45-4f8c-a112-51ac3dbd1271 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0398269 May 24 11:22:25.756607 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:25.866492 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-6e2af873-6134-40bf-91f2-353031da86b6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:26.073488 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 9376988c-0749-43f3-a506-c8c29a15cbcf details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:22:26.208623 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=9376988c-0749-43f3-a506-c8c29a15cbcf,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:22:26.221255 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'port_id': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'mac_address': 'fa:16:3e:aa:5b:b7', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 500, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:22:26.280632 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:26.297616 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 9376988c-0749-43f3-a506-c8c29a15cbcf {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:22:26.313374 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=9376988c-0749-43f3-a506-c8c29a15cbcf,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:22:26.317374 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:26.319276 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:26.384109 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=9376988c-0749-43f3-a506-c8c29a15cbcf,revision_number=9']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:26.391167 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:26.391511 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:26.391844 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:26.392173 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:26.392501 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:26.392826 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:26.393167 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:26.393491 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:26.393814 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:26.394128 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:26.394445 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:26.394761 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:26.399068 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:26.550201 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['9376988c-0749-43f3-a506-c8c29a15cbcf'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:22:26.799178 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-06fbe5c1-7586-478c-9d96-1f343645d368 req-b7991299-d1d0-459a-be62-33c32d8a72e4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:26.800650 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-06fbe5c1-7586-478c-9d96-1f343645d368 req-b7991299-d1d0-459a-be62-33c32d8a72e4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 945 time: 0.0889082 May 24 11:22:26.847827 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-06fbe5c1-7586-478c-9d96-1f343645d368 req-4bb47fc0-a217-421c-9e62-e6200a313bf0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0409484 May 24 11:22:26.974903 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:28.037062 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-afba680c-ccb8-42d1-b6d2-f965265f067b req-b0a7eb82-1033-4a9d-b2e8-892437cb3b68 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:28.038537 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-afba680c-ccb8-42d1-b6d2-f965265f067b req-b0a7eb82-1033-4a9d-b2e8-892437cb3b68 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 945 time: 0.0795999 May 24 11:22:28.073397 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 9376988c-0749-43f3-a506-c8c29a15cbcf up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:22:28.082037 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-afba680c-ccb8-42d1-b6d2-f965265f067b req-abd567be-d5cd-4d97-8e76-2539cac72d38 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0379038 May 24 11:22:28.143525 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 9376988c-0749-43f3-a506-c8c29a15cbcf triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:22:28.143806 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:28.262483 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:28.289249 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:22:28.298400 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=9376988c-0749-43f3-a506-c8c29a15cbcf,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:22:28.302881 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:28.304890 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-d71c3a32-d104-46c1-8f8a-0c765c2e8c4a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:28.342954 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'code': 200} May 24 11:22:28.376150 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=9376988c-0749-43f3-a506-c8c29a15cbcf,revision_number=10']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:28.379107 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:28.867334 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-909efed4-01e1-4f04-aac8-0e1f409a6de1 req-98cf3701-ef8e-43e5-8a2b-fda0e11b8cbb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:28.868773 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-909efed4-01e1-4f04-aac8-0e1f409a6de1 req-98cf3701-ef8e-43e5-8a2b-fda0e11b8cbb service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 1100 time: 0.0717468 May 24 11:22:28.920248 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-909efed4-01e1-4f04-aac8-0e1f409a6de1 req-1e38353a-e43f-434a-a619-d03c39466485 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0421474 May 24 11:22:28.927427 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:29.283394 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6eb9091b-0f53-48b5-bd14-9c8472d036b6 req-4c508b4d-3290-4828-908b-3947412e28e7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:29.284788 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6eb9091b-0f53-48b5-bd14-9c8472d036b6 req-4c508b4d-3290-4828-908b-3947412e28e7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 947 time: 0.0890641 May 24 11:22:29.292018 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:22:29.329815 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6eb9091b-0f53-48b5-bd14-9c8472d036b6 req-3da76573-5d88-493f-a14c-bf3f16f103d0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0390992 May 24 11:22:30.527071 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ed17c4cb-bbfa-446f-894e-a878a1299751 req-28818b74-775b-4a71-9da4-3155784266ae tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:30.528523 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ed17c4cb-bbfa-446f-894e-a878a1299751 req-28818b74-775b-4a71-9da4-3155784266ae tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 947 time: 0.0821152 May 24 11:22:30.575060 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ed17c4cb-bbfa-446f-894e-a878a1299751 req-a6ebc435-58fb-4efb-95e5-1010cc464115 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0409451 May 24 11:22:39.477525 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5155f82a-b36b-4307-892d-6ffb69c53041 req-b5516013-31cc-4d60-9279-8dcc8812f804 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:39.478723 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5155f82a-b36b-4307-892d-6ffb69c53041 req-b5516013-31cc-4d60-9279-8dcc8812f804 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 945 time: 0.0826252 May 24 11:22:39.528056 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5155f82a-b36b-4307-892d-6ffb69c53041 req-96ad6336-c948-44db-ab1e-3029e759895b tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0434976 May 24 11:22:40.718165 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70101b10-d6ff-4e8e-8f0f-bfdbe86aa898 req-27ab0044-73d6-4f95-929f-33c75708e826 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:40.719610 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-70101b10-d6ff-4e8e-8f0f-bfdbe86aa898 req-27ab0044-73d6-4f95-929f-33c75708e826 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 945 time: 0.0848796 May 24 11:22:40.764988 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-70101b10-d6ff-4e8e-8f0f-bfdbe86aa898 req-1eabc95b-f052-4716-a359-7ec3a76a8e83 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0392289 May 24 11:22:40.876457 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad1f5ccd-eba0-41a0-b597-799b1b6ee041 req-c3e5a77a-6688-47f8-8f89-e8034bf8cc4b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:40.877974 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ad1f5ccd-eba0-41a0-b597-799b1b6ee041 req-c3e5a77a-6688-47f8-8f89-e8034bf8cc4b tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 945 time: 0.0807333 May 24 11:22:40.928329 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ad1f5ccd-eba0-41a0-b597-799b1b6ee041 req-82c9dc65-a66f-4f0b-bee6-0f1d91b758e1 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0439205 May 24 11:22:41.089669 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-83a2b5c0-db87-4be9-86b7-11564b16d025 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:41.090975 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-83a2b5c0-db87-4be9-86b7-11564b16d025 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 945 time: 0.1603758 May 24 11:22:41.225456 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Deleting port 8a7c6451-6355-44ad-a36a-f02ffcfe252c {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:22:41.225803 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:41.357028 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] For port 8a7c6451-6355-44ad-a36a-f02ffcfe252c, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=8a7c6451-6355-44ad-a36a-f02ffcfe252c,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:22:41.357339 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:41.645956 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:41.646320 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:41.646632 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Floating IP 6caa87da-7c20-4c7c-a70d-c42930870a74 disassociated. External IP: 172.27.0.211, port: 8a7c6451-6355-44ad-a36a-f02ffcfe252c. May 24 11:22:41.646968 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Calling delete_port for 8a7c6451-6355-44ad-a36a-f02ffcfe252c owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:22:41.756052 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:41.761418 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-756b0c24-ce57-456e-a712-a0b02326bc4b tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:41.810142 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event deleted for resources: {'Port': ['ID=8a7c6451-6355-44ad-a36a-f02ffcfe252c,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:41.821231 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:41.821605 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:41.821970 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:41.822336 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:41.822693 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:41.823049 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:41.823404 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:41.823754 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:41.824102 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:41.824450 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:41.824805 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:41.825158 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:41.828783 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:41.994571 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2f834d23-5fca-44ec-95ca-4fa258d6b63e req-ff6e4ae5-e8d8-4db3-af08-74636f16e661 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 210 time: 0.1182563 May 24 11:22:42.047786 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Router 81705dbf-841d-45b4-8b2d-afa9886a8e59 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:22:42.114120 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f16468eb-fd47-4b44-9268-d9e276359062 req-587e7eb9-776c-4d66-a2cb-ccabaeaa63a3 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:42.115509 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f16468eb-fd47-4b44-9268-d9e276359062 req-587e7eb9-776c-4d66-a2cb-ccabaeaa63a3 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 945 time: 0.0798304 May 24 11:22:42.162519 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f16468eb-fd47-4b44-9268-d9e276359062 req-44d5c871-0ff9-426c-a1b1-708ff81789e7 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0412419 May 24 11:22:42.238016 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:22:42.248653 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7a8d726e-e438-40b0-9088-a99f652f4684 None None] Sync routers for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:22:42.259313 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7a8d726e-e438-40b0-9088-a99f652f4684 None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:22:42.259948 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-7a8d726e-e438-40b0-9088-a99f652f4684 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:22:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:42.311363 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] No DVR routers for this DVR port 8a7c6451-6355-44ad-a36a-f02ffcfe252c on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:22:42.314505 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:42.317295 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'name': 'network-vif-deleted', 'tag': '8a7c6451-6355-44ad-a36a-f02ffcfe252c'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:22:42.334542 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e3bf8afc-e22f-461d-ac9a-7de05aa4a75b req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "DELETE /v2.0/ports/8a7c6451-6355-44ad-a36a-f02ffcfe252c HTTP/1.1" status: 204 len: 173 time: 1.2366359 May 24 11:22:42.372198 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a', 'name': 'network-vif-deleted', 'tag': '8a7c6451-6355-44ad-a36a-f02ffcfe252c', 'status': 'completed', 'code': 200} May 24 11:22:42.495328 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-7a8d726e-e438-40b0-9088-a99f652f4684 None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:22:42.593307 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-5256174d-7132-4775-b889-7480c76374eb tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:42.594746 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-5256174d-7132-4775-b889-7480c76374eb tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.93 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 945 time: 0.0838418 May 24 11:22:42.823129 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-7a8d726e-e438-40b0-9088-a99f652f4684 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:42.826861 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7a8d726e-e438-40b0-9088-a99f652f4684 None None] Checking router: 81705dbf-841d-45b4-8b2d-afa9886a8e59 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:22:42.830251 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Deleting port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:22:42.830908 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:42.839781 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7a8d726e-e438-40b0-9088-a99f652f4684 None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:22:42.840412 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-7a8d726e-e438-40b0-9088-a99f652f4684 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:22:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:42.965909 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] For port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:22:42.966263 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:43.022050 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-7a8d726e-e438-40b0-9088-a99f652f4684 None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:22:43.180885 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2e875c9f-603b-4292-88db-d9464aea5dc9 req-6abd9cb1-e321-4654-82d6-852137313fef tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=7d0e28a2-e3fe-4323-9aba-bb5f4f7d693a HTTP/1.1" status: 200 len: 210 time: 0.0547335 May 24 11:22:43.354735 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-7a8d726e-e438-40b0-9088-a99f652f4684 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:43.357551 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:43.357920 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:43.358231 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Floating IP 5283da39-39a3-4d0a-887a-08f5fdb56c35 disassociated. External IP: 172.27.0.194, port: 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6. May 24 11:22:43.358568 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Calling delete_port for 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:22:43.359011 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7a8d726e-e438-40b0-9088-a99f652f4684 None None] The sync data for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi is: [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:22:43.422640 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d5642125-f3c5-4e4f-ace5-456e8c40cd80 req-19775729-458e-4fd9-bc86-6b3d80ce1935 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:43.424164 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d5642125-f3c5-4e4f-ace5-456e8c40cd80 req-19775729-458e-4fd9-bc86-6b3d80ce1935 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 945 time: 0.1229010 May 24 11:22:43.517877 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d5642125-f3c5-4e4f-ace5-456e8c40cd80 req-e6402ba3-39a4-4272-a5c6-5da4261a74aa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups?id=19f35add-ecda-474f-b1ad-b25cb242f065&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0763099 May 24 11:22:43.552262 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:43.558583 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-81400e2b-df7f-4b4d-9da8-9f0c5a69b33d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:43.609242 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event deleted for resources: {'Port': ['ID=6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:43.613050 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:43.613990 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-01567ac3-8e7c-43fc-afa9-e0e5a330b1cb None None] New status for floating IP 6caa87da-7c20-4c7c-a70d-c42930870a74: DOWN {{(pid=91822) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:22:43.799613 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Router f76568b3-17ef-487b-beeb-3f592a58e138 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:22:43.992796 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:22:44.007624 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7ce3d15c-bd40-457d-aa8b-04f899b3c418 None None] Sync routers for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:22:44.017686 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7ce3d15c-bd40-457d-aa8b-04f899b3c418 None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:22:44.019112 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-7ce3d15c-bd40-457d-aa8b-04f899b3c418 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:22:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:44.077343 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] No DVR routers for this DVR port 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:22:44.081573 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:44.099932 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f12ce744-64b7-4990-9575-49b5ac006ce6 req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.93 "DELETE /v2.0/ports/6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 HTTP/1.1" status: 204 len: 173 time: 1.4995439 May 24 11:22:44.213987 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-7ce3d15c-bd40-457d-aa8b-04f899b3c418 None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:22:44.373162 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'name': 'network-vif-deleted', 'tag': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:22:44.421511 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0ecfad34-cc32-48c2-bffa-73cda667c4ff', 'name': 'network-vif-deleted', 'tag': '6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6', 'status': 'completed', 'code': 200} May 24 11:22:44.554902 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-5de9b09d-1cf8-4291-84f3-3820a34b2252 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-262467324&fields=id&tenant_id=9a8d61f79bbc496ca30e390fd9b5839e HTTP/1.1" status: 200 len: 267 time: 0.0420520 May 24 11:22:44.597200 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-7ce3d15c-bd40-457d-aa8b-04f899b3c418 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:17Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:44.616471 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7ce3d15c-bd40-457d-aa8b-04f899b3c418 None None] Checking router: f76568b3-17ef-487b-beeb-3f592a58e138 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:22:44.626317 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7ce3d15c-bd40-457d-aa8b-04f899b3c418 None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:22:44.627004 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-7ce3d15c-bd40-457d-aa8b-04f899b3c418 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:22:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:44.705914 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a6f10f8b-9260-4066-a82d-d2a639d86718 req-50002047-a3d2-40b6-a71b-631109354a98 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/ports?device_id=0ecfad34-cc32-48c2-bffa-73cda667c4ff HTTP/1.1" status: 200 len: 210 time: 0.0548487 May 24 11:22:44.737638 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-12cf10bc-22bf-4bb7-bf9c-b938e7564e6c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:44.738590 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-12cf10bc-22bf-4bb7-bf9c-b938e7564e6c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Synchronizing usage tracker for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:network {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:22:44.756848 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-12cf10bc-22bf-4bb7-bf9c-b938e7564e6c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Unset dirty status for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:network {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:22:44.758287 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-12cf10bc-22bf-4bb7-bf9c-b938e7564e6c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 811 time: 0.2025604 May 24 11:22:44.773854 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-f9f2e776-a216-4e4f-9949-891628a4492c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/quotas/9a8d61f79bbc496ca30e390fd9b5839e HTTP/1.1" status: 200 len: 375 time: 0.0095747 May 24 11:22:44.830698 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-7ce3d15c-bd40-457d-aa8b-04f899b3c418 None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:22:44.858865 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-9ce735ee-c41a-47fb-abff-62634a8f7e15 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Synchronizing usage tracker for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:22:44.875193 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-9ce735ee-c41a-47fb-abff-62634a8f7e15 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Unset dirty status for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:22:44.876726 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-9ce735ee-c41a-47fb-abff-62634a8f7e15 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&fields=id HTTP/1.1" status: 200 len: 348 time: 0.0974367 May 24 11:22:45.112744 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-13f6108f-e0f7-4092-823a-f9d664e1c90f service neutron] 10.222.0.18 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2242393 May 24 11:22:45.148989 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-7ce3d15c-bd40-457d-aa8b-04f899b3c418 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:17Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:45.153074 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7ce3d15c-bd40-457d-aa8b-04f899b3c418 None None] The sync data for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi is: [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:16:06Z', 'revision_number': 4, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5', 'name': '', 'network_id': '1d92641a-d8f9-4ed6-9abd-f464bb26811d', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'mac_address': 'fa:16:3e:16:09:ee', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_interface', 'standard_attr_id': 80, 'fixed_ips': [{'subnet_id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:16:05Z', 'updated_at': '2021-05-24T11:16:17Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'subnets': [{'id': '2fc4be2e-875c-4d10-b857-a48f90c53cf2', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:22:45.247650 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-909a920e-967d-412a-87b0-05e11217a34f service neutron] 10.222.0.18 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1289973 May 24 11:22:45.372943 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e1dd01ea-9fac-44af-8261-26c000844185 None None] New status for floating IP 5283da39-39a3-4d0a-887a-08f5fdb56c35: DOWN {{(pid=91822) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:22:45.648421 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-826a8d6f-9a24-468b-82aa-25470b170426 req-e6ea1766-dcd9-442d-aaa7-edca8c3f77e0 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 210 time: 0.0445817 May 24 11:22:45.828818 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59550) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:22:45.898580 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:45.929382 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:45.990178 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:45.994123 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-cbebcf4a-61b7-4d38-969c-103a634c289b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:46.008358 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=19f35add-ecda-474f-b1ad-b25cb242f065,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:46.018675 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:46.019055 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:46.019414 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:46.019771 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:46.020124 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:46.020474 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:46.020824 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:46.021200 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:46.021550 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:46.021894 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:46.022239 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:46.022581 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:22:46.026296 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:46.033126 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:46.034987 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "DELETE /v2.0/security-groups/19f35add-ecda-474f-b1ad-b25cb242f065 HTTP/1.1" status: 204 len: 168 time: 0.2050865 May 24 11:22:46.039072 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59556) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:22:46.058702 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.api.v2.resource [None req-95d4c051-de4e-45a6-81a2-5a5ad68f523a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] show failed (client error): The resource could not be found. May 24 11:22:46.059708 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-95d4c051-de4e-45a6-81a2-5a5ad68f523a tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "GET /v2.0/security-groups/19f35add-ecda-474f-b1ad-b25cb242f065 HTTP/1.1" status: 404 len: 347 time: 0.0196497 May 24 11:22:46.062791 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59560) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:22:46.150047 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 8a7c6451-6355-44ad-a36a-f02ffcfe252c no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:22:46.187556 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 8a7c6451-6355-44ad-a36a-f02ffcfe252c {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:22:46.187907 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 8a7c6451-6355-44ad-a36a-f02ffcfe252c not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:22:46.188355 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:22:46.227021 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:22:46.227405 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 6b31c43f-1d25-4fe8-b2c4-56ae3d0252f6 not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:22:46.294814 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-c1a8621c-6fa3-4d61-970f-f9632a8497fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Deleting port 03fa7a36-c8c0-4619-ba8d-280b53b6cfd9 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:22:46.295405 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c1a8621c-6fa3-4d61-970f-f9632a8497fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:46.412515 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c1a8621c-6fa3-4d61-970f-f9632a8497fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:46.412874 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c1a8621c-6fa3-4d61-970f-f9632a8497fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:46.541650 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-c1a8621c-6fa3-4d61-970f-f9632a8497fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Calling delete_port for 03fa7a36-c8c0-4619-ba8d-280b53b6cfd9 owned by network:floatingip {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:22:46.726352 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-a733c42b-621a-42dd-9dce-597b9e582cfe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:46.727799 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-a733c42b-621a-42dd-9dce-597b9e582cfe tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 811 time: 0.1755791 May 24 11:22:46.851386 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d61eb2e9-109e-4f8a-8e75-235548dfd1ed req-b0b19d39-ac3d-4c3c-a6b6-cd4f0b5389c6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 210 time: 0.0675488 May 24 11:22:46.873277 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-af52d313-4e2d-4756-9346-0bc5297cf954 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=9a8d61f79bbc496ca30e390fd9b5839e HTTP/1.1" status: 200 len: 379 time: 0.1367786 May 24 11:22:46.885242 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'port': {'device_id': '45596010-5877-4cfc-8808-752c422ae93a', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'admin_state_up': True, 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'security_groups': ['5928fed2-3207-44a0-91e4-485200bd1a27']}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:22:46.902590 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c1a8621c-6fa3-4d61-970f-f9632a8497fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:46.956631 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8effe5ec-72ba-48d8-9272-9d0724b67711 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:47.038205 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c1a8621c-6fa3-4d61-970f-f9632a8497fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Pushing event deleted for resources: {'Port': ['ID=03fa7a36-c8c0-4619-ba8d-280b53b6cfd9,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:47.049339 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c1a8621c-6fa3-4d61-970f-f9632a8497fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:47.147461 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c1a8621c-6fa3-4d61-970f-f9632a8497fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks [] for floatingip, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:47.162507 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:22:47.177416 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c1a8621c-6fa3-4d61-970f-f9632a8497fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:47.178520 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-c1a8621c-6fa3-4d61-970f-f9632a8497fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] 10.222.0.18 "DELETE /v2.0/floatingips/5283da39-39a3-4d0a-887a-08f5fdb56c35 HTTP/1.1" status: 204 len: 168 time: 1.1148267 May 24 11:22:47.179678 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:22:47.191554 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Made reservation on behalf of 9a8d61f79bbc496ca30e390fd9b5839e for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:22:47.195155 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:47.473578 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Allocated IP 172.20.0.9 (be65d1af-351e-4e5b-8c19-1b04ff5cfc29/994391be-baae-4f60-9ef9-bf02d19d3477/8ca4165e-4c84-4940-9148-3f5a77eaf348) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:22:47.665584 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:47.680802 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Transition to ACTIVE for port object 8ca4165e-4c84-4940-9148-3f5a77eaf348 will not be triggered until provisioned by entity DHCP. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:22:47.703821 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:47.709641 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9bf19145-1f37-4563-8b01-8c317e2cbba1 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:47.775001 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:22:47.777951 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:22:47.781726 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:22:47.818582 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'Port': ['ID=8ca4165e-4c84-4940-9148-3f5a77eaf348,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:47.823034 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:47.851396 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:47.852005 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:47.880641 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.0002170 May 24 11:22:47.895063 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Request body: {'port': {'device_id': '45596010-5877-4cfc-8808-752c422ae93a', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:22:48.011995 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:48.020849 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-945dbdaf-69e5-4026-b9ea-1fcc0096b86f None None] Provisioning for port 8ca4165e-4c84-4940-9148-3f5a77eaf348 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:22:48.026020 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-945dbdaf-69e5-4026-b9ea-1fcc0096b86f None None] Provisioning complete for port 8ca4165e-4c84-4940-9148-3f5a77eaf348 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:22:48.026338 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-945dbdaf-69e5-4026-b9ea-1fcc0096b86f None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:48.058077 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-02f99aad-edc6-4755-81ee-5babba3a45dc req-c34c03cb-2243-4621-94fc-ed1f877779f8 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:48.059499 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-02f99aad-edc6-4755-81ee-5babba3a45dc req-c34c03cb-2243-4621-94fc-ed1f877779f8 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 931 time: 0.0840640 May 24 11:22:48.094693 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-945dbdaf-69e5-4026-b9ea-1fcc0096b86f None None] Port 8ca4165e-4c84-4940-9148-3f5a77eaf348 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:22:48.109321 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-02f99aad-edc6-4755-81ee-5babba3a45dc req-a1ffaccf-d0ca-4729-bae5-e6bebff9a6e9 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0437376 May 24 11:22:48.368159 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 8ca4165e-4c84-4940-9148-3f5a77eaf348 {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:22:48.375182 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:48.390470 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Transition to ACTIVE for port object 8ca4165e-4c84-4940-9148-3f5a77eaf348 will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:22:48.395636 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:48.399200 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:48.575463 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Pushing event updated for resources: {'Port': ['ID=8ca4165e-4c84-4940-9148-3f5a77eaf348,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:48.577870 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.179s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:48.658868 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Attempting to bind port 8ca4165e-4c84-4940-9148-3f5a77eaf348 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:22:48.659173 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Attempting to bind port 8ca4165e-4c84-4940-9148-3f5a77eaf348 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:22:48.659502 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Attempting to bind port 8ca4165e-4c84-4940-9148-3f5a77eaf348 on network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:22:48.664167 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Port 8ca4165e-4c84-4940-9148-3f5a77eaf348 on network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:22:48.664483 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Attempting to bind port 8ca4165e-4c84-4940-9148-3f5a77eaf348 on network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:22:48.669590 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 22, 22), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:22:48.669923 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Checking segment: {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:22:48.670302 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Bound using segment: {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:22:48.671206 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Bound port: 8ca4165e-4c84-4940-9148-3f5a77eaf348, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:22:48.671554 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:48.772427 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] For port 8ca4165e-4c84-4940-9148-3f5a77eaf348, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:22:48.775449 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] For port 8ca4165e-4c84-4940-9148-3f5a77eaf348, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=8ca4165e-4c84-4940-9148-3f5a77eaf348,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:22:48.779071 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:48.783597 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c1a8621c-6fa3-4d61-970f-f9632a8497fd tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:48.841052 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:48.841687 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:48.842992 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] 10.222.0.93 "PUT /v2.0/ports/8ca4165e-4c84-4940-9148-3f5a77eaf348 HTTP/1.1" status: 200 len: 1093 time: 0.9527907 May 24 11:22:48.843490 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '45596010-5877-4cfc-8808-752c422ae93a', 'tag': '8ca4165e-4c84-4940-9148-3f5a77eaf348'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:22:48.869599 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Pushing event updated for resources: {'Port': ['ID=8ca4165e-4c84-4940-9148-3f5a77eaf348,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:48.872900 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:48.922434 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '45596010-5877-4cfc-8808-752c422ae93a', 'tag': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'status': 'completed', 'code': 200} May 24 11:22:48.966849 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-a2ab193e-3cd9-468f-8f8e-b3eafd99763c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:48.968305 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-a2ab193e-3cd9-468f-8f8e-b3eafd99763c service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 1096 time: 0.0802505 May 24 11:22:49.043623 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-93e715b0-0782-42f4-838d-ac2b5dc87fe3 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=8ca4165e-4c84-4940-9148-3f5a77eaf348 HTTP/1.1" status: 200 len: 217 time: 0.0675933 May 24 11:22:49.105944 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-0460245e-9181-43fb-a75a-056b49800c09 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:49.107256 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-0460245e-9181-43fb-a75a-056b49800c09 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.0578616 May 24 11:22:49.182361 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-7c8abaaa-440f-45e0-a107-da6a57108912 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:49.183812 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-7c8abaaa-440f-45e0-a107-da6a57108912 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0700216 May 24 11:22:49.301632 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-00ac3aa1-0fdd-433e-8588-571d3b450737 req-2639d588-66ff-42af-8a13-d6b5f68fd56d tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:49.303277 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-00ac3aa1-0fdd-433e-8588-571d3b450737 req-2639d588-66ff-42af-8a13-d6b5f68fd56d tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 943 time: 0.0891118 May 24 11:22:49.348187 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-00ac3aa1-0fdd-433e-8588-571d3b450737 req-fe0ccf5c-d715-44ac-a6f6-8fe4b62ec158 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0387638 May 24 11:22:49.405816 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-a403aaa7-d54c-4431-8ee0-6996e02fb264 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2158315 May 24 11:22:49.538672 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a9a795b9-346a-41b7-b298-2eafc3c62994 req-c9331108-fb85-4872-8a5b-971253161b6b service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1262374 May 24 11:22:49.663113 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e6c3f10-0161-477a-8769-df0fa9794850 req-6949c251-a5af-41ea-abe1-a2cd65087622 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:49.664656 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5e6c3f10-0161-477a-8769-df0fa9794850 req-6949c251-a5af-41ea-abe1-a2cd65087622 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 1096 time: 0.0771863 May 24 11:22:49.811336 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e6c3f10-0161-477a-8769-df0fa9794850 req-e01d7198-24d1-4db3-9af2-86b944dddf54 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:49.812785 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5e6c3f10-0161-477a-8769-df0fa9794850 req-e01d7198-24d1-4db3-9af2-86b944dddf54 service neutron] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 912 time: 0.1414604 May 24 11:22:49.888408 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5e6c3f10-0161-477a-8769-df0fa9794850 req-3ed5e270-a91b-440a-bcf4-b8ce1860fafc service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=8ca4165e-4c84-4940-9148-3f5a77eaf348 HTTP/1.1" status: 200 len: 217 time: 0.0686171 May 24 11:22:49.960028 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e6c3f10-0161-477a-8769-df0fa9794850 req-ac921fb1-cd1f-4a79-bf69-fe4c1a074a96 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:49.961491 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5e6c3f10-0161-477a-8769-df0fa9794850 req-ac921fb1-cd1f-4a79-bf69-fe4c1a074a96 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.0666361 May 24 11:22:50.039624 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e6c3f10-0161-477a-8769-df0fa9794850 req-bcd93f2b-9dce-4f58-893b-5fab169e8b46 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:50.041077 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5e6c3f10-0161-477a-8769-df0fa9794850 req-bcd93f2b-9dce-4f58-893b-5fab169e8b46 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0718820 May 24 11:22:50.177984 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5e6c3f10-0161-477a-8769-df0fa9794850 req-ce04b6f8-d247-46ea-9627-65d2e7836b33 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1295779 May 24 11:22:50.308888 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5e6c3f10-0161-477a-8769-df0fa9794850 req-923f0335-170d-4dd6-bf2c-6c13d1d79bdf service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1241288 May 24 11:22:50.532691 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2bd0f2e-08d4-4aa7-8e55-e4e88c7c1b7f req-332ed043-333d-4140-ba9e-332607c29e19 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:50.533982 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-d2bd0f2e-08d4-4aa7-8e55-e4e88c7c1b7f req-332ed043-333d-4140-ba9e-332607c29e19 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Synchronizing usage tracker for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:22:50.550478 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-d2bd0f2e-08d4-4aa7-8e55-e4e88c7c1b7f req-332ed043-333d-4140-ba9e-332607c29e19 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Unset dirty status for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:22:50.552205 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d2bd0f2e-08d4-4aa7-8e55-e4e88c7c1b7f req-332ed043-333d-4140-ba9e-332607c29e19 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 943 time: 0.1006548 May 24 11:22:50.600480 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d2bd0f2e-08d4-4aa7-8e55-e4e88c7c1b7f req-c49c8c44-c9cc-41a1-a18f-6e3fa4253366 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0427089 May 24 11:22:51.220281 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59600) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:22:51.791250 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-26a15921-70dc-43fc-88b4-0d71e8eeefa7 req-96558d68-4015-4f82-8de6-51a6678556b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:51.792702 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-26a15921-70dc-43fc-88b4-0d71e8eeefa7 req-96558d68-4015-4f82-8de6-51a6678556b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 943 time: 0.0834773 May 24 11:22:51.844512 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-26a15921-70dc-43fc-88b4-0d71e8eeefa7 req-45361588-8c01-4fc4-8ce7-2a57f3834d21 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0448217 May 24 11:22:51.952351 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-687465'] for router_interface, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:52.038105 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:52.364091 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] Deleting port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:22:52.365345 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:52.484128 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] For port e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:22:52.484480 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:52.608126 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] Calling delete_port for e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 owned by network:router_interface {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:22:52.749584 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:52.753726 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8471bb97-6200-418f-8d87-21a2714ae1cc service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:52.795573 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] Pushing event deleted for resources: {'Port': ['ID=e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:52.799189 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:53.120503 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-537767', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-685561'] for router_interface, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:53.282883 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-026641d0-65b3-466d-a71b-16fb540b4804 req-0e338a75-30d6-40d7-a12f-e045627bc7f4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:53.284819 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-026641d0-65b3-466d-a71b-16fb540b4804 req-0e338a75-30d6-40d7-a12f-e045627bc7f4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 943 time: 0.2786462 May 24 11:22:53.405914 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-026641d0-65b3-466d-a71b-16fb540b4804 req-987b8c9b-6bcc-4dd4-99c5-6a76bf78a78d tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0689726 May 24 11:22:53.420629 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a6fd13ce-c425-473f-b274-637bfd329d68 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:53.612365 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] Router f76568b3-17ef-487b-beeb-3f592a58e138 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:22:53.786598 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:22:53.795619 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-78284fe8-0feb-4d5b-9993-0d9334634638 None None] Sync routers for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:22:53.798549 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] 10.222.0.18 "PUT /v2.0/routers/f76568b3-17ef-487b-beeb-3f592a58e138/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.5770752 May 24 11:22:53.801198 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59612) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:22:53.805453 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-78284fe8-0feb-4d5b-9993-0d9334634638 None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:22:53.806077 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-78284fe8-0feb-4d5b-9993-0d9334634638 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:22:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:53.978564 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Notify callbacks [] for router, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:53.983692 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-78284fe8-0feb-4d5b-9993-0d9334634638 None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:22:54.157494 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:54.230323 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Deleting port 541360c9-23fa-4c8b-9a48-116e140aa6a4 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:22:54.230655 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:54.270050 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-78284fe8-0feb-4d5b-9993-0d9334634638 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39'}], 'enable_snat': True}, 'gw_port_id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:22:52Z', 'revision_number': 5, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port': {'id': '541360c9-23fa-4c8b-9a48-116e140aa6a4', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:b4:93:4c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.39', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:15:58Z', 'updated_at': '2021-05-24T11:16:08Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:54.273903 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-78284fe8-0feb-4d5b-9993-0d9334634638 None None] Checking router: f76568b3-17ef-487b-beeb-3f592a58e138 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:22:54.282262 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-78284fe8-0feb-4d5b-9993-0d9334634638 None None] Routers {'f76568b3-17ef-487b-beeb-3f592a58e138'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:22:54.283397 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-78284fe8-0feb-4d5b-9993-0d9334634638 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:22:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['f76568b3-17ef-487b-beeb-3f592a58e138'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:54.352289 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] For port 541360c9-23fa-4c8b-9a48-116e140aa6a4, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=541360c9-23fa-4c8b-9a48-116e140aa6a4,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:22:54.352655 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:54.475095 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Calling delete_port for 541360c9-23fa-4c8b-9a48-116e140aa6a4 owned by network:router_gateway {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:22:54.475710 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-78284fe8-0feb-4d5b-9993-0d9334634638 None None] Set router f76568b3-17ef-487b-beeb-3f592a58e138 gateway port host: None {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:22:54.594204 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-78284fe8-0feb-4d5b-9993-0d9334634638 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:22:54Z', 'revision_number': 6, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port_host': None}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:54.599148 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-78284fe8-0feb-4d5b-9993-0d9334634638 None None] The sync data for ids ['f76568b3-17ef-487b-beeb-3f592a58e138'] in dv-cinder-784092-9-iscsi is: [{'id': 'f76568b3-17ef-487b-beeb-3f592a58e138', 'name': 'tempest-AttachVolumeTestJSON-1010114930-router', 'tenant_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:15:57Z', 'updated_at': '2021-05-24T11:22:54Z', 'revision_number': 6, 'project_id': 'b303f83a84794e5ba6a1bab9c76b429c', 'gw_port_host': None}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:22:54.628724 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7b6a7640-b2b9-49c3-a81f-43d62ab59821 req-5ee2c2bb-07fb-4937-a924-5a0260633572 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:54.630373 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7b6a7640-b2b9-49c3-a81f-43d62ab59821 req-5ee2c2bb-07fb-4937-a924-5a0260633572 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 943 time: 0.1107264 May 24 11:22:54.679783 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7b6a7640-b2b9-49c3-a81f-43d62ab59821 req-b928ee8e-b129-4d23-9b11-4861268903b1 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0426776 May 24 11:22:54.744582 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:54.748159 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:54.793923 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Pushing event deleted for resources: {'Port': ['ID=541360c9-23fa-4c8b-9a48-116e140aa6a4,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:54.797037 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:54.888545 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-686964'] for router_gateway, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.068949 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-684537', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4171'] for router, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.071162 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.155602 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-684229'] for router, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.156204 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Fanout notify agent at l3_agent the message router_deleted on router f76568b3-17ef-487b-beeb-3f592a58e138 {{(pid=91819) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 11:22:55.171211 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Notify callbacks [] for router, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.172343 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] 10.222.0.18 "DELETE /v2.0/routers/f76568b3-17ef-487b-beeb-3f592a58e138 HTTP/1.1" status: 204 len: 168 time: 1.3702476 May 24 11:22:55.175685 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59618) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:22:55.233997 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_v2 [None req-b9322231-d082-409b-9432-0023605f7c60 admin admin] Deleting subnet 2fc4be2e-875c-4d10-b857-a48f90c53cf2 {{(pid=91819) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:22:55.236810 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b9322231-d082-409b-9432-0023605f7c60 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.285400 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b9322231-d082-409b-9432-0023605f7c60 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534853172'] for subnet, precommit_delete_associations {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.402924 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b9322231-d082-409b-9432-0023605f7c60 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.654694 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b9322231-d082-409b-9432-0023605f7c60 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534944900', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535570805'] for subnet, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.659092 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:55.690889 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b9322231-d082-409b-9432-0023605f7c60 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=2fc4be2e-875c-4d10-b857-a48f90c53cf2,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:55.694255 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b9322231-d082-409b-9432-0023605f7c60 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:55.696912 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b9322231-d082-409b-9432-0023605f7c60 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.702376 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c62d84bc-7326-41e9-8b28-7229c28b14ad None None] Network 1d92641a-d8f9-4ed6-9abd-f464bb26811d requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:22:55.709419 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-b9322231-d082-409b-9432-0023605f7c60 admin admin] 10.222.0.18 "DELETE /v2.0/subnets/2fc4be2e-875c-4d10-b857-a48f90c53cf2 HTTP/1.1" status: 204 len: 168 time: 0.5306151 May 24 11:22:55.709736 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59624) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:22:55.793550 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.877237 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.897929 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8202f0c8-ab39-4828-a255-0443e1f8d476 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:55.955546 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f71c6fc-16d9-48bc-8b48-5ce17220c35a req-faa957a8-f3dc-4fd0-849c-03a661f72176 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:55.955546 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6f71c6fc-16d9-48bc-8b48-5ce17220c35a req-faa957a8-f3dc-4fd0-849c-03a661f72176 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 943 time: 0.1528659 May 24 11:22:55.989899 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-079942fb-d8ed-44df-8cf6-989bee881074 None None] DHCP port deletion for 1d92641a-d8f9-4ed6-9abd-f464bb26811d request from dv-cinder-784092-9-iscsi {{(pid=91821) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 11:22:56.013423 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6f71c6fc-16d9-48bc-8b48-5ce17220c35a req-e6f532c1-5852-474d-8c6a-00c81d25185a tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0523567 May 24 11:22:56.018849 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Deleting port 0377cd0a-2401-44a9-a8d2-cc3ce593483b {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:22:56.019386 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.054409 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-079942fb-d8ed-44df-8cf6-989bee881074 None None] Deleting port 0377cd0a-2401-44a9-a8d2-cc3ce593483b {{(pid=91821) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:22:56.054709 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-079942fb-d8ed-44df-8cf6-989bee881074 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.118390 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] For port 0377cd0a-2401-44a9-a8d2-cc3ce593483b, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=0377cd0a-2401-44a9-a8d2-cc3ce593483b,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:22:56.118742 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.154497 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-079942fb-d8ed-44df-8cf6-989bee881074 None None] For port 0377cd0a-2401-44a9-a8d2-cc3ce593483b, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=0377cd0a-2401-44a9-a8d2-cc3ce593483b,segment=NetworkSegment(bbef7673-75cf-415f-81a6-941c270eac8b),segment_id=bbef7673-75cf-415f-81a6-941c270eac8b)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:22:56.154887 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-079942fb-d8ed-44df-8cf6-989bee881074 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.279139 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-079942fb-d8ed-44df-8cf6-989bee881074 None None] Calling delete_port for 0377cd0a-2401-44a9-a8d2-cc3ce593483b owned by network:dhcp {{(pid=91821) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:22:56.325614 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-079942fb-d8ed-44df-8cf6-989bee881074 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093'] for port, after_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.329164 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:56.335183 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Calling delete_port for 0377cd0a-2401-44a9-a8d2-cc3ce593483b owned by network:dhcp {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:22:56.371994 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-079942fb-d8ed-44df-8cf6-989bee881074 None None] Pushing event deleted for resources: {'Port': ['ID=0377cd0a-2401-44a9-a8d2-cc3ce593483b,revision_number=None']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:56.374998 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-079942fb-d8ed-44df-8cf6-989bee881074 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:56.379756 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.380609 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-079942fb-d8ed-44df-8cf6-989bee881074 None None] No DVR routers for this DVR port 0377cd0a-2401-44a9-a8d2-cc3ce593483b on host dv-cinder-784092-9-iscsi {{(pid=91821) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:22:56.383614 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:56.426964 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Pushing event deleted for resources: {'Port': ['ID=0377cd0a-2401-44a9-a8d2-cc3ce593483b,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:56.430089 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:56.430860 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] No DVR routers for this DVR port 0377cd0a-2401-44a9-a8d2-cc3ce593483b on host dv-cinder-784092-9-iscsi {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:22:56.535364 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 541360c9-23fa-4c8b-9a48-116e140aa6a4 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:22:56.548521 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.562068 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['1d92641a-d8f9-4ed6-9abd-f464bb26811d']}} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:56.570076 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 541360c9-23fa-4c8b-9a48-116e140aa6a4 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:22:56.570430 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 541360c9-23fa-4c8b-9a48-116e140aa6a4 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:22:56.570853 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 0377cd0a-2401-44a9-a8d2-cc3ce593483b no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:22:56.574325 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'bbef7673-75cf-415f-81a6-941c270eac8b') {'for_net_delete': True} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:56.574620 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'bbef7673-75cf-415f-81a6-941c270eac8b') {} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:22:56.586397 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534853183'] for segment, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.602061 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.603907 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 0377cd0a-2401-44a9-a8d2-cc3ce593483b {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:22:56.604397 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 0377cd0a-2401-44a9-a8d2-cc3ce593483b not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:22:56.604815 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:22:56.608635 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Releasing vlan 518 on physical network physnet1 to pool {{(pid=91819) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:22:56.609043 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535155490'] for segment, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: /usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py:1359: SAWarning: DELETE statement on table 'portsecuritybindings' expected to delete 1 row(s); 0 were matched. Please set confirm_deleted_rows=False within the mapper configuration to prevent this warning. May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: util.warn( May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: /usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py:1359: SAWarning: DELETE statement on table 'ml2_port_binding_levels' expected to delete 1 row(s); 0 were matched. Please set confirm_deleted_rows=False within the mapper configuration to prevent this warning. May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: util.warn( May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: /usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py:1359: SAWarning: DELETE statement on table 'ml2_port_bindings' expected to delete 1 row(s); 0 were matched. Please set confirm_deleted_rows=False within the mapper configuration to prevent this warning. May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: util.warn( May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: /usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py:1359: SAWarning: DELETE statement on table 'ports' expected to delete 1 row(s); 0 were matched. Please set confirm_deleted_rows=False within the mapper configuration to prevent this warning. May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: util.warn( May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: Traceback (most recent call last): May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: timer() May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: cb(*args, **kw) May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: return func(*args, **kwargs) May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: self._notify() May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: self.callback(batched_events) May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: event.method(event) May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: return f(self, *a, **k) May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._get(url).json() May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: return self.request(url, 'GET', **kwargs) May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:22:56.627456 dv-cinder-784092-9-iscsi neutron-server[91819]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:22:56.642794 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:22:56.643080 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device e1261c4d-d7c7-4c3e-ae8a-372bcc316ae5 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:22:56.700970 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.705280 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2c179284-c9f4-4f68-b4ba-f9924ce65927 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:56.721224 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.730631 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Pushing event deleted for resources: {'Network': ['ID=1d92641a-d8f9-4ed6-9abd-f464bb26811d,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:56.734483 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:56.743347 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] 10.222.0.18 "DELETE /v2.0/networks/1d92641a-d8f9-4ed6-9abd-f464bb26811d HTTP/1.1" status: 204 len: 168 time: 1.0326300 May 24 11:22:56.855492 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59630) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:22:56.906639 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-73d758c5-3baf-431a-bf2a-ef9a603609c6 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=b303f83a84794e5ba6a1bab9c76b429c&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0498266 May 24 11:22:56.909565 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59634) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:22:56.967147 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f3809d68-d533-4f3e-9315-5ef58dda4bb0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.981231 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:56.994938 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f3809d68-d533-4f3e-9315-5ef58dda4bb0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:57.046401 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f3809d68-d533-4f3e-9315-5ef58dda4bb0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:57.052095 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a9536fa8-185d-47eb-b959-f1de451f3aef admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:57.066301 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f3809d68-d533-4f3e-9315-5ef58dda4bb0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f9e46c40-bf1d-4484-bb18-341b351d3f29,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:57.068693 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f3809d68-d533-4f3e-9315-5ef58dda4bb0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:57.074454 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-f3809d68-d533-4f3e-9315-5ef58dda4bb0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:57.075557 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-f3809d68-d533-4f3e-9315-5ef58dda4bb0 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/f9e46c40-bf1d-4484-bb18-341b351d3f29 HTTP/1.1" status: 204 len: 168 time: 0.1650918 May 24 11:22:57.206501 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e95264ea-87ff-45f5-9751-8e394028a4c9 req-30eaa3ce-855b-4dfe-9a25-f3007afc4c18 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:57.207981 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e95264ea-87ff-45f5-9751-8e394028a4c9 req-30eaa3ce-855b-4dfe-9a25-f3007afc4c18 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 943 time: 0.0859787 May 24 11:22:57.258131 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e95264ea-87ff-45f5-9751-8e394028a4c9 req-4d5e2ef5-237d-4c72-9057-0f412b466bfc tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0441816 May 24 11:22:58.225421 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 8ca4165e-4c84-4940-9148-3f5a77eaf348 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:22:58.398128 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 8ca4165e-4c84-4940-9148-3f5a77eaf348, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=8ca4165e-4c84-4940-9148-3f5a77eaf348,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:22:58.413933 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'port_id': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'mac_address': 'fa:16:3e:a6:bc:9e', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 503, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:22:58.479631 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:58.499562 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 8ca4165e-4c84-4940-9148-3f5a77eaf348 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:22:58.508210 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38e6e98c-cf11-420a-89c4-c00877322040 req-e42a9721-d795-42bb-819d-00b24dfb7b8a tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:58.509754 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-38e6e98c-cf11-420a-89c4-c00877322040 req-e42a9721-d795-42bb-819d-00b24dfb7b8a tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 943 time: 0.0983109 May 24 11:22:58.519358 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 8ca4165e-4c84-4940-9148-3f5a77eaf348, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=8ca4165e-4c84-4940-9148-3f5a77eaf348,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:22:58.526422 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:58.528349 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:22:58.562708 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-38e6e98c-cf11-420a-89c4-c00877322040 req-1cc66908-d680-40a7-b953-c895a0db93b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0460720 May 24 11:22:58.598613 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=8ca4165e-4c84-4940-9148-3f5a77eaf348,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:22:58.601508 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:22:58.744750 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['8ca4165e-4c84-4940-9148-3f5a77eaf348'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:22:58.976820 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:22:59.757235 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-975f0542-ea36-4b5d-8836-5a9f76987d5b req-508f2ea2-139a-483a-8483-74f9746fd032 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:22:59.758868 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-975f0542-ea36-4b5d-8836-5a9f76987d5b req-508f2ea2-139a-483a-8483-74f9746fd032 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 944 time: 0.0877914 May 24 11:22:59.807465 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-975f0542-ea36-4b5d-8836-5a9f76987d5b req-66b5ad0a-dfb8-430f-9971-b008ad75bb65 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0414596 May 24 11:23:00.994938 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f30e6ea3-95aa-4c77-be9c-09deef12303d req-5f99e266-e9b6-4924-baf3-26b01cfd41bf tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:00.996364 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f30e6ea3-95aa-4c77-be9c-09deef12303d req-5f99e266-e9b6-4924-baf3-26b01cfd41bf tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 944 time: 0.0830157 May 24 11:23:01.040342 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f30e6ea3-95aa-4c77-be9c-09deef12303d req-fee89d6c-f7b8-45fc-8346-eff95fc42578 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0374572 May 24 11:23:01.209627 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 8ca4165e-4c84-4940-9148-3f5a77eaf348 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:23:01.279887 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 8ca4165e-4c84-4940-9148-3f5a77eaf348 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:23:01.284857 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 8ca4165e-4c84-4940-9148-3f5a77eaf348 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:23:01.285266 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:01.395848 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:01.421710 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '45596010-5877-4cfc-8808-752c422ae93a', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '8ca4165e-4c84-4940-9148-3f5a77eaf348'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:23:01.432348 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 8ca4165e-4c84-4940-9148-3f5a77eaf348, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=8ca4165e-4c84-4940-9148-3f5a77eaf348,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:23:01.436306 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:01.438661 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:01.470678 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '45596010-5877-4cfc-8808-752c422ae93a', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'code': 200} May 24 11:23:01.507261 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=8ca4165e-4c84-4940-9148-3f5a77eaf348,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:01.510098 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:02.226086 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6da78380-6a55-45d4-8121-8cca6b0d913d req-c31bfca3-9bfb-4e14-ac22-55891fa2ca86 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:02.227572 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6da78380-6a55-45d4-8121-8cca6b0d913d req-c31bfca3-9bfb-4e14-ac22-55891fa2ca86 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 945 time: 0.0803876 May 24 11:23:02.276941 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6da78380-6a55-45d4-8121-8cca6b0d913d req-24fad9b2-5fcc-4f5f-baec-dc3632bd1ff5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0438280 May 24 11:23:03.482749 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-20468a42-6618-4f90-912d-ec3b6f8488e7 req-25d7e873-0720-4d70-a141-a9bf1b2cebfb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:03.484521 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-20468a42-6618-4f90-912d-ec3b6f8488e7 req-25d7e873-0720-4d70-a141-a9bf1b2cebfb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 945 time: 0.0961454 May 24 11:23:03.530860 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-20468a42-6618-4f90-912d-ec3b6f8488e7 req-b0b5c883-86f5-4ef0-8bb4-d66349c0fab4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0405207 May 24 11:23:03.603116 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5d528a38-0f35-43be-86b1-46eb3eaf1926 req-fe460dc7-2780-4247-91e4-4e7494afe1c8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:03.604972 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5d528a38-0f35-43be-86b1-46eb3eaf1926 req-fe460dc7-2780-4247-91e4-4e7494afe1c8 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 1098 time: 0.0709863 May 24 11:23:03.647592 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5d528a38-0f35-43be-86b1-46eb3eaf1926 req-17d8b230-e60b-4cdd-8055-e0cf49a9d889 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0344315 May 24 11:23:04.734703 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6ad5ea14-e64d-447c-a677-c48e499b6cfd req-c43485c4-f0e9-4475-aa73-be862b3bd371 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:04.736154 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6ad5ea14-e64d-447c-a677-c48e499b6cfd req-c43485c4-f0e9-4475-aa73-be862b3bd371 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 945 time: 0.0850105 May 24 11:23:04.779928 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6ad5ea14-e64d-447c-a677-c48e499b6cfd req-84e61c55-4c3e-435a-8fca-6e31e18acb90 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0380054 May 24 11:23:04.969054 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-932a56a7-3963-46eb-bfdb-da668a8b3d4d req-5b3aa0de-d1d0-4190-af69-70da4d28aa14 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:04.970535 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-932a56a7-3963-46eb-bfdb-da668a8b3d4d req-5b3aa0de-d1d0-4190-af69-70da4d28aa14 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 945 time: 0.0865915 May 24 11:23:05.018693 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-932a56a7-3963-46eb-bfdb-da668a8b3d4d req-14cc29cf-ab48-4c7f-ab05-e7d3227240d2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0424542 May 24 11:23:05.041595 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59750) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:05.114147 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ee6bd238-38b7-41d7-87ab-f7955c985020 tempest-TestVolumeBootPattern-1615708037 tempest-TestVolumeBootPattern-1615708037-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:05.115644 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-ee6bd238-38b7-41d7-87ab-f7955c985020 tempest-TestVolumeBootPattern-1615708037 tempest-TestVolumeBootPattern-1615708037-project-admin] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 1093 time: 0.0730591 May 24 11:23:05.118471 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59754) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:05.123456 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'floatingip': {'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'port_id': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address': '172.20.0.9'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:23:05.129363 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:05.133265 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Usage tracker for resource:floatingip and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:05.136220 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Quota usage for floatingip was recalculated. Used quota:1. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:05.142706 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource floatingip. Total usage: 1; quota limit: 50; headroom:49 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:05.154093 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:05.295216 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:05.438877 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Allocated IP 172.27.0.8 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/332cf5a0-e4e4-4e30-be84-a0e470103da6) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:23:05.592151 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:05.622890 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:05.627401 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-63110852-77cd-4a0e-b1f1-213cda1060f8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:05.693979 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:23:05.697425 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:23:05.700590 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:23:05.730935 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'Port': ['ID=332cf5a0-e4e4-4e30-be84-a0e470103da6,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:05.738961 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:06.222152 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7fae2366-e65b-4dc6-948c-4204818997b8 req-7b344e4e-5543-468e-b661-536bc53c99f4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:06.223595 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7fae2366-e65b-4dc6-948c-4204818997b8 req-7b344e4e-5543-468e-b661-536bc53c99f4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 947 time: 0.1899273 May 24 11:23:06.292515 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7fae2366-e65b-4dc6-948c-4204818997b8 req-761d8e80-e371-4f6e-9369-968e0962b744 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0588772 May 24 11:23:06.294104 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:23:06.315113 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:06.390302 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:06.619403 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:06.625785 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:06.629464 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-bc9fe0d9-c749-4ba5-9b49-994a23c67963 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:06.737029 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'Port': ['ID=332cf5a0-e4e4-4e30-be84-a0e470103da6,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:06.741356 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:06.752894 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:06.929458 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Floating IP bfd25b24-9190-4f5b-be56-6f7de757deb6 associated. External IP: 172.27.0.8, port: 8ca4165e-4c84-4940-9148-3f5a77eaf348. May 24 11:23:07.222763 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59792) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:07.286299 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Router 81705dbf-841d-45b4-8b2d-afa9886a8e59 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:23:07.343462 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-85325d48-07ef-4710-96ad-0bfe451fa324 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:07.360100 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-85325d48-07ef-4710-96ad-0bfe451fa324 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:07.365846 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-85325d48-07ef-4710-96ad-0bfe451fa324 admin admin] Usage tracker for resource:security_group_rule and tenant:134156ea4c634568982f0dbb6dbccf06 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:07.368964 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-85325d48-07ef-4710-96ad-0bfe451fa324 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:07.375744 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-85325d48-07ef-4710-96ad-0bfe451fa324 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:07.472462 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-85325d48-07ef-4710-96ad-0bfe451fa324 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:07.476688 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-85325d48-07ef-4710-96ad-0bfe451fa324 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:07.484424 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-81166790-def8-4626-ad3d-59f016bada5a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:07.525726 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-85325d48-07ef-4710-96ad-0bfe451fa324 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=134156ea4c634568982f0dbb6dbccf06&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3018782 May 24 11:23:07.526695 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-85325d48-07ef-4710-96ad-0bfe451fa324 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=226a2ef9-7d1c-4d1a-ade3-fc9691b2d73f,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:07.528989 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59800) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:07.531448 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-85325d48-07ef-4710-96ad-0bfe451fa324 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:07.592642 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-14e49825-57ab-4c88-9774-70e46042e3f7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:07.618746 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-14e49825-57ab-4c88-9774-70e46042e3f7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:07.674917 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-14e49825-57ab-4c88-9774-70e46042e3f7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:07.680338 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-df822461-dc26-411c-96dc-8da6142346e4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:07.691345 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-14e49825-57ab-4c88-9774-70e46042e3f7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=226a2ef9-7d1c-4d1a-ade3-fc9691b2d73f,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:07.694801 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-14e49825-57ab-4c88-9774-70e46042e3f7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:07.698257 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-14e49825-57ab-4c88-9774-70e46042e3f7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:07.701599 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-14e49825-57ab-4c88-9774-70e46042e3f7 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/226a2ef9-7d1c-4d1a-ade3-fc9691b2d73f HTTP/1.1" status: 204 len: 168 time: 0.1715944 May 24 11:23:07.707563 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:23:07.722551 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fedd2377-a730-4ba0-afc5-fcb5371d1ab7 None None] Sync routers for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:23:07.734865 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fedd2377-a730-4ba0-afc5-fcb5371d1ab7 None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:23:07.735456 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-fedd2377-a730-4ba0-afc5-fcb5371d1ab7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:22:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:23:07.738063 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7796a008-4592-4409-b2c9-7c7d900eef90 req-f0e4c558-dd23-4f4f-9b3a-0d41080eb60c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:07.739568 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7796a008-4592-4409-b2c9-7c7d900eef90 req-f0e4c558-dd23-4f4f-9b3a-0d41080eb60c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 947 time: 0.2733097 May 24 11:23:07.748122 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-c69168aa-381d-4c48-a26f-dc43ef9c72f3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:07.750950 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-c69168aa-381d-4c48-a26f-dc43ef9c72f3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.93 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 947 time: 0.1795781 May 24 11:23:07.817623 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:07.908021 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7796a008-4592-4409-b2c9-7c7d900eef90 req-45570489-900a-4f0e-bfdf-de0edbd9f793 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=eaab864c-81a2-4598-945c-15413cce7a89&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1531453 May 24 11:23:07.945218 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 994 time: 2.8252869 May 24 11:23:07.950005 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '45596010-5877-4cfc-8808-752c422ae93a', 'tag': '8ca4165e-4c84-4940-9148-3f5a77eaf348'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:23:08.000725 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Deleting port 9376988c-0749-43f3-a506-c8c29a15cbcf {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:23:08.000725 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:08.005493 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-fedd2377-a730-4ba0-afc5-fcb5371d1ab7 None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:23:08.023009 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '45596010-5877-4cfc-8808-752c422ae93a', 'tag': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'status': 'completed', 'code': 200} May 24 11:23:08.124167 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce7fe2ef-0964-4f1d-b061-a6c98363ee5b req-389b62cc-3866-4f6c-84ee-4b2546fa8e5c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:08.126744 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce7fe2ef-0964-4f1d-b061-a6c98363ee5b req-389b62cc-3866-4f6c-84ee-4b2546fa8e5c service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 1098 time: 0.0948105 May 24 11:23:08.162440 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] For port 9376988c-0749-43f3-a506-c8c29a15cbcf, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=9376988c-0749-43f3-a506-c8c29a15cbcf,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:23:08.162909 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:08.393689 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce7fe2ef-0964-4f1d-b061-a6c98363ee5b req-c67393a6-5292-4025-b07a-5cbb6c4e0ee9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:08.395854 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce7fe2ef-0964-4f1d-b061-a6c98363ee5b req-c67393a6-5292-4025-b07a-5cbb6c4e0ee9 service neutron] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 912 time: 0.2622535 May 24 11:23:08.467333 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-fedd2377-a730-4ba0-afc5-fcb5371d1ab7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': 'bfd25b24-9190-4f5b-be56-6f7de757deb6', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.8', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 534, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:a6:bc:9e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '45596010-5877-4cfc-8808-752c422ae93a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:23:06Z', 'updated_at': '2021-05-24T11:23:06Z', 'revision_number': 0, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:23:08.472096 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fedd2377-a730-4ba0-afc5-fcb5371d1ab7 None None] Checking router: 81705dbf-841d-45b4-8b2d-afa9886a8e59 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:23:08.481794 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fedd2377-a730-4ba0-afc5-fcb5371d1ab7 None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:23:08.482575 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-fedd2377-a730-4ba0-afc5-fcb5371d1ab7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:22:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:23:08.515411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:08.516976 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:08.517493 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Floating IP 2f97fb4c-710a-4b28-a681-dfc09baedda8 disassociated. External IP: 172.27.0.70, port: 9376988c-0749-43f3-a506-c8c29a15cbcf. May 24 11:23:08.517895 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Calling delete_port for 9376988c-0749-43f3-a506-c8c29a15cbcf owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:23:08.519931 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce7fe2ef-0964-4f1d-b061-a6c98363ee5b req-57bf97ce-b0c2-49cf-b81d-ac5a031998bb service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=8ca4165e-4c84-4940-9148-3f5a77eaf348 HTTP/1.1" status: 200 len: 997 time: 0.1176670 May 24 11:23:08.607820 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce7fe2ef-0964-4f1d-b061-a6c98363ee5b req-3b85fb58-2e4c-4c73-a95b-874b3bb1f981 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:08.610801 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce7fe2ef-0964-4f1d-b061-a6c98363ee5b req-3b85fb58-2e4c-4c73-a95b-874b3bb1f981 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.0835958 May 24 11:23:08.668446 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:08.675720 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-12d0d622-e8e2-48f9-bbe2-02c61293f96d tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:08.697345 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce7fe2ef-0964-4f1d-b061-a6c98363ee5b req-5fd3cb26-adbb-4209-ab24-ed1810fe8265 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:08.699521 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce7fe2ef-0964-4f1d-b061-a6c98363ee5b req-5fd3cb26-adbb-4209-ab24-ed1810fe8265 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0821586 May 24 11:23:08.702778 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-fedd2377-a730-4ba0-afc5-fcb5371d1ab7 None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:23:08.724257 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event deleted for resources: {'Port': ['ID=9376988c-0749-43f3-a506-c8c29a15cbcf,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:08.728582 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:08.891077 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce7fe2ef-0964-4f1d-b061-a6c98363ee5b req-0e81d9b8-56be-4588-94f3-3017f1eed0d6 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1838813 May 24 11:23:09.054964 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Router c2ffed60-7ad5-4215-9e01-680707fb98d6 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:23:09.248856 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ce7fe2ef-0964-4f1d-b061-a6c98363ee5b req-9a2c13b1-56cc-468e-bad3-92000aa5e282 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3509045 May 24 11:23:09.266383 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-fedd2377-a730-4ba0-afc5-fcb5371d1ab7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': 'bfd25b24-9190-4f5b-be56-6f7de757deb6', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.8', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 534, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:a6:bc:9e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '45596010-5877-4cfc-8808-752c422ae93a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:23:06Z', 'updated_at': '2021-05-24T11:23:06Z', 'revision_number': 0, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:23:09.271675 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fedd2377-a730-4ba0-afc5-fcb5371d1ab7 None None] The sync data for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi is: [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': 'bfd25b24-9190-4f5b-be56-6f7de757deb6', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.8', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 534, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:a6:bc:9e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '45596010-5877-4cfc-8808-752c422ae93a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:23:06Z', 'updated_at': '2021-05-24T11:23:06Z', 'revision_number': 0, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:23:09.447250 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d59a7691-4304-4170-82b4-62ebc6002f63 req-339a6e99-2d3f-4e22-900e-73e8e81b9dd5 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=e201e1c1-db4c-4741-9f38-15502d43246f HTTP/1.1" status: 200 len: 210 time: 0.0732491 May 24 11:23:09.519043 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:23:09.529484 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6cd20ffc-13e3-4c1e-b201-2e8c33a35554 None None] Sync routers for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:23:09.551020 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6cd20ffc-13e3-4c1e-b201-2e8c33a35554 None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:23:09.551867 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-6cd20ffc-13e3-4c1e-b201-2e8c33a35554 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:22:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:23:09.566917 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-341510cf-370c-4b58-b72b-6031d0f03993 None None] New status for floating IP bfd25b24-9190-4f5b-be56-6f7de757deb6: ACTIVE {{(pid=91822) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:23:09.647002 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] No DVR routers for this DVR port 9376988c-0749-43f3-a506-c8c29a15cbcf on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:23:09.650463 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:09.670282 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-660119f4-3f17-4245-8e72-6afa2f7e156e req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.93 "DELETE /v2.0/ports/9376988c-0749-43f3-a506-c8c29a15cbcf HTTP/1.1" status: 204 len: 173 time: 1.9108417 May 24 11:23:09.777364 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-6cd20ffc-13e3-4c1e-b201-2e8c33a35554 None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:23:10.024509 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'name': 'network-vif-deleted', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:23:10.034488 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59850) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:10.079486 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e201e1c1-db4c-4741-9f38-15502d43246f', 'name': 'network-vif-deleted', 'tag': '9376988c-0749-43f3-a506-c8c29a15cbcf', 'status': 'completed', 'code': 200} May 24 11:23:10.145306 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-6cd20ffc-13e3-4c1e-b201-2e8c33a35554 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:26Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:23:10.150412 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6cd20ffc-13e3-4c1e-b201-2e8c33a35554 None None] Checking router: c2ffed60-7ad5-4215-9e01-680707fb98d6 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:23:10.158520 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6cd20ffc-13e3-4c1e-b201-2e8c33a35554 None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:23:10.159136 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-6cd20ffc-13e3-4c1e-b201-2e8c33a35554 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:22:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:23:10.161392 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cb6bb085-c1fa-42f0-9df9-87def7d76ff6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:10.175755 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-cb6bb085-c1fa-42f0-9df9-87def7d76ff6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:10.179537 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-cb6bb085-c1fa-42f0-9df9-87def7d76ff6 admin admin] Usage tracker for resource:security_group_rule and tenant:dbb31e130e3f4d38bc2cf1062b31f066 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:10.182563 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-cb6bb085-c1fa-42f0-9df9-87def7d76ff6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:10.188933 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-cb6bb085-c1fa-42f0-9df9-87def7d76ff6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:10.273292 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cb6bb085-c1fa-42f0-9df9-87def7d76ff6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:10.277358 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-cb6bb085-c1fa-42f0-9df9-87def7d76ff6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:10.291531 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-185c34e5-068c-499d-9a07-ef54406cb1b8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:10.332082 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-cb6bb085-c1fa-42f0-9df9-87def7d76ff6 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=dbb31e130e3f4d38bc2cf1062b31f066&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2960079 May 24 11:23:10.333844 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cb6bb085-c1fa-42f0-9df9-87def7d76ff6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=10789ede-170d-4e31-9b54-34b3d3af130f,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:10.334733 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59856) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:10.337023 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-cb6bb085-c1fa-42f0-9df9-87def7d76ff6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:10.341498 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-6cd20ffc-13e3-4c1e-b201-2e8c33a35554 None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:23:10.400424 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4f38df4e-88e3-47bd-a025-e10001548c5e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:10.427945 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4f38df4e-88e3-47bd-a025-e10001548c5e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:10.484554 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4f38df4e-88e3-47bd-a025-e10001548c5e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:10.491134 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-89b315fa-7ca4-4254-bc0c-fed062d227ee admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:10.503662 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f38df4e-88e3-47bd-a025-e10001548c5e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=10789ede-170d-4e31-9b54-34b3d3af130f,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:10.506290 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4f38df4e-88e3-47bd-a025-e10001548c5e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:10.513608 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-4f38df4e-88e3-47bd-a025-e10001548c5e admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:10.517860 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-4f38df4e-88e3-47bd-a025-e10001548c5e admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/10789ede-170d-4e31-9b54-34b3d3af130f HTTP/1.1" status: 204 len: 168 time: 0.1789970 May 24 11:23:10.589591 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59866) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:10.665400 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-62805f11-21a9-461f-baed-016c6670aac7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:10.695814 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-62805f11-21a9-461f-baed-016c6670aac7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:10.731639 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-6cd20ffc-13e3-4c1e-b201-2e8c33a35554 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:26Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:23:10.736375 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6cd20ffc-13e3-4c1e-b201-2e8c33a35554 None None] The sync data for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi is: [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:26Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:23:10.758948 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-62805f11-21a9-461f-baed-016c6670aac7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:10.765655 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ac23e5ce-6147-4227-a154-0ce403c5a19f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:10.784580 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-62805f11-21a9-461f-baed-016c6670aac7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=eaab864c-81a2-4598-945c-15413cce7a89,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:10.787290 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-62805f11-21a9-461f-baed-016c6670aac7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:10.812564 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-62805f11-21a9-461f-baed-016c6670aac7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:10.814386 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-62805f11-21a9-461f-baed-016c6670aac7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "DELETE /v2.0/security-groups/eaab864c-81a2-4598-945c-15413cce7a89 HTTP/1.1" status: 204 len: 168 time: 0.2237020 May 24 11:23:10.817505 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59872) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:10.840522 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.api.v2.resource [None req-590cecd3-b4f5-4f19-b447-0e9f5a97608d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] show failed (client error): The resource could not be found. May 24 11:23:10.841788 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-590cecd3-b4f5-4f19-b447-0e9f5a97608d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups/eaab864c-81a2-4598-945c-15413cce7a89 HTTP/1.1" status: 404 len: 347 time: 0.0233049 May 24 11:23:10.845478 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59876) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:10.895655 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59880) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:10.916159 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6fca1510-0217-4a4f-8239-8ef1c4575074 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:10.932166 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-6fca1510-0217-4a4f-8239-8ef1c4575074 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:10.937364 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-6fca1510-0217-4a4f-8239-8ef1c4575074 admin admin] Usage tracker for resource:security_group_rule and tenant:7c24498a6dce4dedb2259a3603bd482a is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:10.940338 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-6fca1510-0217-4a4f-8239-8ef1c4575074 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:10.946969 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-6fca1510-0217-4a4f-8239-8ef1c4575074 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:11.044808 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6fca1510-0217-4a4f-8239-8ef1c4575074 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:11.052003 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6fca1510-0217-4a4f-8239-8ef1c4575074 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:11.053808 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-709fccce-e5f6-4184-a173-a6552f188d69 None None] New status for floating IP 2f97fb4c-710a-4b28-a681-dfc09baedda8: DOWN {{(pid=91821) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:23:11.060698 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e023b9ef-bb7a-43cb-82f5-cc4cc363c8d7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:11.086455 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-b75cdd68-8033-489f-91cc-e080dd76b336 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Deleting port 43cda913-ddcb-408c-bbe1-fb1518ca5b28 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:23:11.087028 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b75cdd68-8033-489f-91cc-e080dd76b336 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:11.104341 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-6fca1510-0217-4a4f-8239-8ef1c4575074 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=7c24498a6dce4dedb2259a3603bd482a&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2050979 May 24 11:23:11.104676 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59884) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:11.107839 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6fca1510-0217-4a4f-8239-8ef1c4575074 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5198d898-3112-4968-bc9f-307f926d7b82,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:11.110379 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6fca1510-0217-4a4f-8239-8ef1c4575074 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:11.254795 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ba6cedee-b0cd-475b-86ad-4150326ad012 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:11.266964 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b75cdd68-8033-489f-91cc-e080dd76b336 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:11.267154 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b75cdd68-8033-489f-91cc-e080dd76b336 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:11.407082 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-b75cdd68-8033-489f-91cc-e080dd76b336 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Calling delete_port for 43cda913-ddcb-408c-bbe1-fb1518ca5b28 owned by network:floatingip {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:23:11.410469 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ba6cedee-b0cd-475b-86ad-4150326ad012 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:11.484794 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ba6cedee-b0cd-475b-86ad-4150326ad012 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:11.489109 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-612a357f-460b-4cb8-8ecd-f454527e9970 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:11.501170 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ba6cedee-b0cd-475b-86ad-4150326ad012 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5198d898-3112-4968-bc9f-307f926d7b82,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:11.504968 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ba6cedee-b0cd-475b-86ad-4150326ad012 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:11.544244 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ba6cedee-b0cd-475b-86ad-4150326ad012 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:11.545382 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-ba6cedee-b0cd-475b-86ad-4150326ad012 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/5198d898-3112-4968-bc9f-307f926d7b82 HTTP/1.1" status: 204 len: 168 time: 0.4397202 May 24 11:23:11.574107 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b75cdd68-8033-489f-91cc-e080dd76b336 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:11.579479 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-521dd897-ab3b-4ff9-89b0-e0650b8dcba5 tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:11.627002 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b75cdd68-8033-489f-91cc-e080dd76b336 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event deleted for resources: {'Port': ['ID=43cda913-ddcb-408c-bbe1-fb1518ca5b28,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:11.661549 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b75cdd68-8033-489f-91cc-e080dd76b336 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:11.720236 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b75cdd68-8033-489f-91cc-e080dd76b336 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:11.735264 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b75cdd68-8033-489f-91cc-e080dd76b336 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:11.736380 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-b75cdd68-8033-489f-91cc-e080dd76b336 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "DELETE /v2.0/floatingips/2f97fb4c-710a-4b28-a681-dfc09baedda8 HTTP/1.1" status: 204 len: 168 time: 0.8897498 May 24 11:23:12.069641 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59902) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:12.075331 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Request body: {'security_group': {'name': 'tempest-securitygroup--1991942290', 'description': 'tempest-description--612821910'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:23:12.084269 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:12.085835 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Usage tracker for resource:security_group and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:12.088803 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:12.094976 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:12.106181 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.130592 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:12.134578 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Usage tracker for resource:security_group_rule and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:12.137421 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:12.143121 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:12.219544 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.223643 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.227521 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ff5871d6-0ebc-42e5-b289-24b848a112a8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:12.267513 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event updated for resources: {'SecurityGroup': ['ID=07d7df74-cc03-44ab-b535-c3ef7f5a7611,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:12.272944 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:12.280271 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.282409 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-ba00b6ae-3448-4060-b642-163ab517af99 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1779 time: 0.2112880 May 24 11:23:12.285165 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 59914) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:12.290414 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Request body: {'security_group_rule': {'security_group_id': '07d7df74-cc03-44ab-b535-c3ef7f5a7611', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:23:12.296178 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:12.300932 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Usage tracker for resource:security_group_rule and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:12.302946 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 9376988c-0749-43f3-a506-c8c29a15cbcf no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:23:12.304032 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:12.310345 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:12.329489 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.342376 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 9376988c-0749-43f3-a506-c8c29a15cbcf {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:23:12.343020 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 9376988c-0749-43f3-a506-c8c29a15cbcf not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:23:12.400988 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.404932 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.407430 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:12.430537 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=5c28372f-64fb-4478-bf4c-e663b6e1899d,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:12.433010 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:12.441800 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.442852 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.1567378 May 24 11:23:12.445663 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59920) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:12.450749 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-9103b156-09b8-4abd-8bcd-768c15dbac8d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Request body: {'security_group_rule': {'security_group_id': '07d7df74-cc03-44ab-b535-c3ef7f5a7611', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:23:12.456540 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-9103b156-09b8-4abd-8bcd-768c15dbac8d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:12.460865 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-9103b156-09b8-4abd-8bcd-768c15dbac8d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Usage tracker for resource:security_group_rule and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:12.464019 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-9103b156-09b8-4abd-8bcd-768c15dbac8d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:12.471079 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-9103b156-09b8-4abd-8bcd-768c15dbac8d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:12.490692 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9103b156-09b8-4abd-8bcd-768c15dbac8d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.557508 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9103b156-09b8-4abd-8bcd-768c15dbac8d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.561473 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9103b156-09b8-4abd-8bcd-768c15dbac8d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.563534 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:12.586762 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9103b156-09b8-4abd-8bcd-768c15dbac8d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=98c8491e-c6eb-4656-ad49-c7c9d3b5b3a7,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:12.589269 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9103b156-09b8-4abd-8bcd-768c15dbac8d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:12.597652 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9103b156-09b8-4abd-8bcd-768c15dbac8d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.598831 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-9103b156-09b8-4abd-8bcd-768c15dbac8d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.1521900 May 24 11:23:12.602447 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 59928) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:12.607560 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Request body: {'floatingip': {'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:23:12.615159 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:12.619442 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Usage tracker for resource:floatingip and tenant:05ec02ff35d2402e8d7bf064752e67ea is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:12.622599 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:12.629910 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:12.641453 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.784611 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:12.925070 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Allocated IP 172.27.0.143 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/8ab34aa2-04db-4ff9-88aa-b55bd6070a66) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:23:13.193311 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:13.224893 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:13.232654 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:13.316656 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:23:13.319821 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:23:13.322336 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:23:13.361293 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event updated for resources: {'Port': ['ID=8ab34aa2-04db-4ff9-88aa-b55bd6070a66,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:13.364098 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.132s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:13.415339 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:13.491489 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:13.705004 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:13.710316 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:13.714448 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b451a798-254d-4311-a387-f6978442f7b3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:13.815785 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event updated for resources: {'Port': ['ID=8ab34aa2-04db-4ff9-88aa-b55bd6070a66,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:13.823834 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:13.834149 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:13.857195 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:13.858420 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 693 time: 1.2550142 May 24 11:23:14.108610 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-bb3459bd-6497-40ae-8f10-337d550adb07 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Synchronizing usage tracker for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:security_group {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:23:14.125459 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-bb3459bd-6497-40ae-8f10-337d550adb07 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Unset dirty status for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:security_group {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:23:14.127263 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-bb3459bd-6497-40ae-8f10-337d550adb07 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?name=tempest-securitygroup--1991942290&fields=id&tenant_id=05ec02ff35d2402e8d7bf064752e67ea HTTP/1.1" status: 200 len: 267 time: 0.0580313 May 24 11:23:14.279163 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-d65a29a8-a80e-433d-a7ca-7fa29f5714f0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:14.280594 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-d65a29a8-a80e-433d-a7ca-7fa29f5714f0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 816 time: 0.1469760 May 24 11:23:14.295669 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-5dd35453-657b-4ea1-8c80-62dbbb432f4f tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/quotas/05ec02ff35d2402e8d7bf064752e67ea HTTP/1.1" status: 200 len: 375 time: 0.0089269 May 24 11:23:14.380384 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-42a15ed5-0f3d-44d3-86ea-c62e14deee62 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&fields=id HTTP/1.1" status: 200 len: 348 time: 0.0795150 May 24 11:23:14.630591 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-1b9a44f2-d87e-4ecc-aa4b-0fde275000dc service neutron] 10.222.0.18 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2381525 May 24 11:23:14.788525 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-9a7465d1-b54f-4fe8-b403-8cab745a394d service neutron] 10.222.0.18 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1466722 May 24 11:23:14.978071 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-4bd74529-02c6-4ce9-b77b-ab44f61eeee2 req-d3fbf860-0676-42ac-92f4-5b3b3762d183 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Synchronizing usage tracker for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:23:15.012771 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-4bd74529-02c6-4ce9-b77b-ab44f61eeee2 req-d3fbf860-0676-42ac-92f4-5b3b3762d183 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Unset dirty status for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:23:15.015945 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4bd74529-02c6-4ce9-b77b-ab44f61eeee2 req-d3fbf860-0676-42ac-92f4-5b3b3762d183 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 210 time: 0.0814304 May 24 11:23:16.155546 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-232539a2-02d7-4a5b-bb80-26fe20c978d5 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:16.158254 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-232539a2-02d7-4a5b-bb80-26fe20c978d5 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.93 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 816 time: 0.2122414 May 24 11:23:16.191062 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8b26423a-8a51-4eee-8e72-94747d1bcfd2 req-6e35b742-5632-44be-b9de-f5b570dec257 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 210 time: 0.0555708 May 24 11:23:16.222903 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-7b77ac46-2499-4148-967b-29d7b71238ca tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.93 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=05ec02ff35d2402e8d7bf064752e67ea HTTP/1.1" status: 200 len: 380 time: 0.0561073 May 24 11:23:16.242101 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Request body: {'port': {'device_id': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'admin_state_up': True, 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'security_groups': ['07d7df74-cc03-44ab-b535-c3ef7f5a7611']}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:23:16.387393 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:16.399295 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:16.409703 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Made reservation on behalf of 05ec02ff35d2402e8d7bf064752e67ea for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:23:16.413104 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:16.581172 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Allocated IP 172.20.0.4 (7877e643-fe9e-4041-91fa-3a4f44279fd2/12ff35e8-9af8-4339-95c9-c62c90d8c6fd/a92e9868-2cc3-4d24-859b-18840b6fce56) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:23:16.774015 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:16.789763 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Transition to ACTIVE for port object a92e9868-2cc3-4d24-859b-18840b6fce56 will not be triggered until provisioned by entity DHCP. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:23:16.818234 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:16.825397 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-7e1f88c0-ac85-4973-b705-44bbf4954b18 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:16.889059 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Network 7877e643-fe9e-4041-91fa-3a4f44279fd2 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:23:16.892334 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:23:16.894782 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:23:16.948205 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event updated for resources: {'Port': ['ID=a92e9868-2cc3-4d24-859b-18840b6fce56,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:16.951242 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:16.982344 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:16.982990 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:17.011771 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 0.7766495 May 24 11:23:17.024237 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Request body: {'port': {'device_id': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:23:17.137888 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:17.164044 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-11ebdb2c-6cdf-4aa6-ab44-352fedaa1790 None None] Provisioning for port a92e9868-2cc3-4d24-859b-18840b6fce56 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:23:17.168716 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-11ebdb2c-6cdf-4aa6-ab44-352fedaa1790 None None] Provisioning complete for port a92e9868-2cc3-4d24-859b-18840b6fce56 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:23:17.169109 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-11ebdb2c-6cdf-4aa6-ab44-352fedaa1790 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:17.227190 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-11ebdb2c-6cdf-4aa6-ab44-352fedaa1790 None None] Port a92e9868-2cc3-4d24-859b-18840b6fce56 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:23:17.572786 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2ee92d5-534f-4552-859f-6cd641500892 req-2605526b-8fe7-4f2a-b5bb-9e17306ccfbc tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:17.573497 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-d2ee92d5-534f-4552-859f-6cd641500892 req-2605526b-8fe7-4f2a-b5bb-9e17306ccfbc tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Synchronizing usage tracker for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:23:17.640795 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-d2ee92d5-534f-4552-859f-6cd641500892 req-2605526b-8fe7-4f2a-b5bb-9e17306ccfbc tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Unset dirty status for tenant:05ec02ff35d2402e8d7bf064752e67ea on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:23:17.641964 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d2ee92d5-534f-4552-859f-6cd641500892 req-2605526b-8fe7-4f2a-b5bb-9e17306ccfbc tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 931 time: 0.2032981 May 24 11:23:17.664155 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id a92e9868-2cc3-4d24-859b-18840b6fce56 {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:23:17.675982 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:17.707595 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Transition to ACTIVE for port object a92e9868-2cc3-4d24-859b-18840b6fce56 will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:23:17.712633 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d2ee92d5-534f-4552-859f-6cd641500892 req-059db173-3002-4c46-be93-63a9bc5d9c04 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0642946 May 24 11:23:17.713650 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:17.718016 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e727e511-a615-48d8-8d36-ae5f0f629858 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:17.871195 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Pushing event updated for resources: {'Port': ['ID=a92e9868-2cc3-4d24-859b-18840b6fce56,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:17.879212 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.161s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:17.952609 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Attempting to bind port a92e9868-2cc3-4d24-859b-18840b6fce56 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:23:17.952952 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Attempting to bind port a92e9868-2cc3-4d24-859b-18840b6fce56 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:23:17.953314 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Attempting to bind port a92e9868-2cc3-4d24-859b-18840b6fce56 on network 7877e643-fe9e-4041-91fa-3a4f44279fd2 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:23:17.957991 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Port a92e9868-2cc3-4d24-859b-18840b6fce56 on network 7877e643-fe9e-4041-91fa-3a4f44279fd2 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:23:17.958310 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Attempting to bind port a92e9868-2cc3-4d24-859b-18840b6fce56 on network 7877e643-fe9e-4041-91fa-3a4f44279fd2 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:23:17.963309 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 22, 52), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:23:17.963643 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Checking segment: {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:23:17.964019 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Bound using segment: {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:23:17.964446 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Bound port: a92e9868-2cc3-4d24-859b-18840b6fce56, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '31ae7943-96d7-4018-b76d-057ae76567e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:23:17.964790 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:18.055536 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:23:18.058733 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=a92e9868-2cc3-4d24-859b-18840b6fce56,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:23:18.064317 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:18.069580 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a91ab59f-d2c2-47f8-8b81-0cbdd43a2ba4 tempest-ServerRescueNegativeTestJSON-1983762590 tempest-ServerRescueNegativeTestJSON-1983762590-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:18.124717 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:18.125497 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:18.126920 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] 10.222.0.93 "PUT /v2.0/ports/a92e9868-2cc3-4d24-859b-18840b6fce56 HTTP/1.1" status: 200 len: 1093 time: 1.1071298 May 24 11:23:18.127438 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:23:18.159984 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Pushing event updated for resources: {'Port': ['ID=a92e9868-2cc3-4d24-859b-18840b6fce56,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:18.163201 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:18.184653 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'status': 'completed', 'code': 200} May 24 11:23:18.962293 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88ebc3c0-401e-4cbd-9787-60aafbec8296 req-16343a0c-d544-44a0-8a51-4e00e3555c28 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:18.963755 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-88ebc3c0-401e-4cbd-9787-60aafbec8296 req-16343a0c-d544-44a0-8a51-4e00e3555c28 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.0951278 May 24 11:23:19.007861 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-88ebc3c0-401e-4cbd-9787-60aafbec8296 req-db127e9f-84bd-444a-845d-c20c195efc0b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0376453 May 24 11:23:20.005189 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-98abbd03-9232-4505-940c-e2d8c7785678 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:20.006656 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-98abbd03-9232-4505-940c-e2d8c7785678 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 1096 time: 0.0771623 May 24 11:23:20.085519 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-f12e3acd-a067-4319-89ad-e8c225dc5415 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=a92e9868-2cc3-4d24-859b-18840b6fce56 HTTP/1.1" status: 200 len: 217 time: 0.0717299 May 24 11:23:20.156859 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-95f02983-e7a4-4f6c-a319-77a012e0baaf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:20.158840 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-95f02983-e7a4-4f6c-a319-77a012e0baaf service neutron] 10.222.0.93 "GET /v2.0/subnets?id=12ff35e8-9af8-4339-95c9-c62c90d8c6fd HTTP/1.1" status: 200 len: 855 time: 0.0659385 May 24 11:23:20.224334 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9fb4ecd9-b449-48ba-ab42-a0c562805505 req-abb77006-fbd0-4af3-8611-27b2e85180e4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:20.225625 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9fb4ecd9-b449-48ba-ab42-a0c562805505 req-abb77006-fbd0-4af3-8611-27b2e85180e4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.0934021 May 24 11:23:20.268107 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-e3abec9a-1256-4a70-84ef-df091091baab service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:20.269874 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-e3abec9a-1256-4a70-84ef-df091091baab service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=7877e643-fe9e-4041-91fa-3a4f44279fd2&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1019814 May 24 11:23:20.285155 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9fb4ecd9-b449-48ba-ab42-a0c562805505 req-401482b1-2e1b-4018-8ea5-ce7aa531d004 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0477321 May 24 11:23:20.413422 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-f9a79c60-77af-4560-9cc3-cb87d76fc00f service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1363769 May 24 11:23:20.580801 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-87dfcabc-706a-4f7a-97ba-273fdb10969b req-349b1c33-2ed3-406a-8264-f5e293130731 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1607139 May 24 11:23:20.696883 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f71c885-8ef9-4ff6-8b70-656c6413480d req-90ae594f-e85b-4611-8425-770d59bf985b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:20.698424 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2f71c885-8ef9-4ff6-8b70-656c6413480d req-90ae594f-e85b-4611-8425-770d59bf985b service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 1096 time: 0.0739326 May 24 11:23:20.845320 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f71c885-8ef9-4ff6-8b70-656c6413480d req-cc71b542-b661-479f-9c1d-ff735166a63d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:20.846850 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2f71c885-8ef9-4ff6-8b70-656c6413480d req-cc71b542-b661-479f-9c1d-ff735166a63d service neutron] 10.222.0.93 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 917 time: 0.1412635 May 24 11:23:21.023973 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2f71c885-8ef9-4ff6-8b70-656c6413480d req-7c90774c-1db2-447d-b0f3-b009b18709a4 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=a92e9868-2cc3-4d24-859b-18840b6fce56 HTTP/1.1" status: 200 len: 217 time: 0.1708360 May 24 11:23:21.089940 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f71c885-8ef9-4ff6-8b70-656c6413480d req-f7121582-48fb-474b-ab2e-b8b16ca77fa4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:21.091271 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2f71c885-8ef9-4ff6-8b70-656c6413480d req-f7121582-48fb-474b-ab2e-b8b16ca77fa4 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=12ff35e8-9af8-4339-95c9-c62c90d8c6fd HTTP/1.1" status: 200 len: 855 time: 0.0600977 May 24 11:23:21.174028 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f71c885-8ef9-4ff6-8b70-656c6413480d req-9260d894-d29c-41ee-92da-42189bb20eff service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:21.175862 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2f71c885-8ef9-4ff6-8b70-656c6413480d req-9260d894-d29c-41ee-92da-42189bb20eff service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=7877e643-fe9e-4041-91fa-3a4f44279fd2&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0773535 May 24 11:23:21.311773 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2f71c885-8ef9-4ff6-8b70-656c6413480d req-058f23c0-c42e-4789-b92b-71694f81c5cc service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1288047 May 24 11:23:21.510002 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2f71c885-8ef9-4ff6-8b70-656c6413480d req-d4d8e79b-5f70-490d-bec2-780f5e2613e3 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1905186 May 24 11:23:21.532290 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-652904a4-c975-446b-891b-0743f2f7c1a3 req-bc7fe5eb-ac1f-4610-aa8c-7da1441608b3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:21.533795 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-652904a4-c975-446b-891b-0743f2f7c1a3 req-bc7fe5eb-ac1f-4610-aa8c-7da1441608b3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.1225727 May 24 11:23:21.581446 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-652904a4-c975-446b-891b-0743f2f7c1a3 req-2d91caf4-cf9e-4977-a527-549d09d876d7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0420802 May 24 11:23:22.052011 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:22.883204 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1c3169ab-b7c5-45c8-8793-e2c187a47c03 req-0e5f7c2e-1538-4aba-bd66-74fb259f855e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:22.884653 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1c3169ab-b7c5-45c8-8793-e2c187a47c03 req-0e5f7c2e-1538-4aba-bd66-74fb259f855e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.1817956 May 24 11:23:22.894445 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device a92e9868-2cc3-4d24-859b-18840b6fce56 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:23:22.931732 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1c3169ab-b7c5-45c8-8793-e2c187a47c03 req-2e5543c1-22c1-407e-92e9-59cd9de70993 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0408556 May 24 11:23:23.033432 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=a92e9868-2cc3-4d24-859b-18840b6fce56,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:23:23.046531 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'port_id': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'mac_address': 'fa:16:3e:48:62:19', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 500, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.4'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:23:23.101417 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:23.118280 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id a92e9868-2cc3-4d24-859b-18840b6fce56 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:23:23.136997 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=a92e9868-2cc3-4d24-859b-18840b6fce56,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:23:23.141418 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:23.143174 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:23.212230 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=a92e9868-2cc3-4d24-859b-18840b6fce56,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:23.215214 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:23.368409 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['a92e9868-2cc3-4d24-859b-18840b6fce56'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:23:23.431587 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-4d187140-20f7-4024-b1aa-5b379bf64c63 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:23.983389 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 60084) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:24.121915 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b0790238-3dfa-47aa-97c1-ed81516f73b5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:24.145189 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-b0790238-3dfa-47aa-97c1-ed81516f73b5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:24.150578 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-b0790238-3dfa-47aa-97c1-ed81516f73b5 admin admin] Usage tracker for resource:security_group_rule and tenant:86e7d9f5c1044027bfb99b81dc5336c2 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:24.155486 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-b0790238-3dfa-47aa-97c1-ed81516f73b5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:24.163664 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-b0790238-3dfa-47aa-97c1-ed81516f73b5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:24.193565 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f01ef9c-2a15-4eed-a48d-d88eae16a508 req-6e21f8cf-148e-48a6-8524-a9d9d6e5fb13 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:24.195000 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6f01ef9c-2a15-4eed-a48d-d88eae16a508 req-6e21f8cf-148e-48a6-8524-a9d9d6e5fb13 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 944 time: 0.1392627 May 24 11:23:24.258363 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6f01ef9c-2a15-4eed-a48d-d88eae16a508 req-8fec46f0-32b7-4cb6-96f5-19f836a63c0d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0549359 May 24 11:23:24.314780 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b0790238-3dfa-47aa-97c1-ed81516f73b5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:24.319670 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-b0790238-3dfa-47aa-97c1-ed81516f73b5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:24.326445 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f3809d68-d533-4f3e-9315-5ef58dda4bb0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:24.368240 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-b0790238-3dfa-47aa-97c1-ed81516f73b5 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=86e7d9f5c1044027bfb99b81dc5336c2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3830440 May 24 11:23:24.368672 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b0790238-3dfa-47aa-97c1-ed81516f73b5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=460f6d19-3b67-4874-8882-5af329d60687,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:24.370935 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 60090) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:24.371847 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b0790238-3dfa-47aa-97c1-ed81516f73b5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:24.430040 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1591872-ec09-4928-bf0e-bf973af49d07 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:24.455698 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1591872-ec09-4928-bf0e-bf973af49d07 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:24.510019 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1591872-ec09-4928-bf0e-bf973af49d07 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:24.514362 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-73bda772-80e8-4b18-a001-d3d225095b31 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:24.527891 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e1591872-ec09-4928-bf0e-bf973af49d07 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=460f6d19-3b67-4874-8882-5af329d60687,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:24.531739 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e1591872-ec09-4928-bf0e-bf973af49d07 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:24.534982 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e1591872-ec09-4928-bf0e-bf973af49d07 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:24.536073 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-e1591872-ec09-4928-bf0e-bf973af49d07 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/460f6d19-3b67-4874-8882-5af329d60687 HTTP/1.1" status: 204 len: 168 time: 0.1638720 May 24 11:23:24.832097 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 60098) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:24.851526 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-86007b7a-e0db-4075-8d08-7c4b144047f5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:24.866155 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-86007b7a-e0db-4075-8d08-7c4b144047f5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:24.870267 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-86007b7a-e0db-4075-8d08-7c4b144047f5 admin admin] Usage tracker for resource:security_group_rule and tenant:6e012eec67fc422cad6b7914b29cef69 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:24.873371 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-86007b7a-e0db-4075-8d08-7c4b144047f5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:24.878896 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-86007b7a-e0db-4075-8d08-7c4b144047f5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:24.910804 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device a92e9868-2cc3-4d24-859b-18840b6fce56 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:23:24.970544 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-86007b7a-e0db-4075-8d08-7c4b144047f5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:24.974903 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-86007b7a-e0db-4075-8d08-7c4b144047f5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:24.978516 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port a92e9868-2cc3-4d24-859b-18840b6fce56 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:23:24.983219 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port a92e9868-2cc3-4d24-859b-18840b6fce56 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:23:24.983596 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:24.986834 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0cb4cb40-4044-4268-8674-89e7f450fa67 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:25.024556 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-86007b7a-e0db-4075-8d08-7c4b144047f5 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=6e012eec67fc422cad6b7914b29cef69&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1891534 May 24 11:23:25.025332 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 60102) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:25.038236 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-86007b7a-e0db-4075-8d08-7c4b144047f5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=52626941-592f-4ac5-bc46-2e596d55f1ca,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:25.041230 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-86007b7a-e0db-4075-8d08-7c4b144047f5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:25.088549 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-12d08499-4ac9-4916-8727-b79db0fe13ab admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:25.115136 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-12d08499-4ac9-4916-8727-b79db0fe13ab admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:25.118319 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:25.148107 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:23:25.159645 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=a92e9868-2cc3-4d24-859b-18840b6fce56,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:23:25.163763 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:25.166100 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:25.192276 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-12d08499-4ac9-4916-8727-b79db0fe13ab admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:25.200072 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-00ebade1-f758-486e-a9d4-325450be4759 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:25.205159 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'code': 200} May 24 11:23:25.213979 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-12d08499-4ac9-4916-8727-b79db0fe13ab admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=52626941-592f-4ac5-bc46-2e596d55f1ca,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:25.217642 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-12d08499-4ac9-4916-8727-b79db0fe13ab admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:25.224117 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-12d08499-4ac9-4916-8727-b79db0fe13ab admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:25.229572 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-12d08499-4ac9-4916-8727-b79db0fe13ab admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/52626941-592f-4ac5-bc46-2e596d55f1ca HTTP/1.1" status: 204 len: 168 time: 0.2032878 May 24 11:23:25.268305 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=a92e9868-2cc3-4d24-859b-18840b6fce56,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:25.272669 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:25.520571 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c0c40232-d954-447e-b942-d897bf73a616 req-3d19dbd5-2ec9-4522-84d6-09a1f12c735a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:25.522072 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c0c40232-d954-447e-b942-d897bf73a616 req-3d19dbd5-2ec9-4522-84d6-09a1f12c735a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.1470411 May 24 11:23:25.548408 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49508a72-31da-4b0f-82f1-3db9aadb2b57 req-7b6c6a86-3c4d-4d11-bccb-50fafd4d3655 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:25.550075 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-49508a72-31da-4b0f-82f1-3db9aadb2b57 req-7b6c6a86-3c4d-4d11-bccb-50fafd4d3655 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 1098 time: 0.1382117 May 24 11:23:25.605686 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c0c40232-d954-447e-b942-d897bf73a616 req-b5945ae8-6808-477c-bee9-99d16694960b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0751793 May 24 11:23:25.612066 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-49508a72-31da-4b0f-82f1-3db9aadb2b57 req-9ad841b1-e1e9-4ddc-b42a-dd3d4eb8070b service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0536265 May 24 11:23:25.747192 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:25.870120 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c775c7c2-19df-4b1f-a9c3-d2ec01d2b1cc None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:26.815658 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-65b4addc-d0c9-4898-99f6-73e23b54701c req-9c46c100-8410-4990-a86b-0e8c3b4b7591 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:26.818734 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-65b4addc-d0c9-4898-99f6-73e23b54701c req-9c46c100-8410-4990-a86b-0e8c3b4b7591 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0958083 May 24 11:23:26.885707 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-65b4addc-d0c9-4898-99f6-73e23b54701c req-d490b107-d06e-4d5b-abe6-addfbe4fd504 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0610125 May 24 11:23:26.989934 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:27.099168 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4b6c4327-bade-42ce-a813-44d87e5f61f5 req-fc0a36c7-37d2-404a-b791-cad4fea88a46 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:27.100676 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4b6c4327-bade-42ce-a813-44d87e5f61f5 req-fc0a36c7-37d2-404a-b791-cad4fea88a46 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0961807 May 24 11:23:27.109581 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 60158) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:27.116916 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-d634024c-b245-4d3e-abc2-ec42809edbfe tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Request body: {'floatingip': {'port_id': 'a92e9868-2cc3-4d24-859b-18840b6fce56'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:23:27.195423 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d634024c-b245-4d3e-abc2-ec42809edbfe tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:27.953710 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d634024c-b245-4d3e-abc2-ec42809edbfe tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:27.957864 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d634024c-b245-4d3e-abc2-ec42809edbfe tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:28.156297 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [None req-d634024c-b245-4d3e-abc2-ec42809edbfe tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Floating IP 1f2903d8-b12d-4d40-bb9f-84aa6c3e7266 associated. External IP: 172.27.0.143, port: a92e9868-2cc3-4d24-859b-18840b6fce56. May 24 11:23:28.514951 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d634024c-b245-4d3e-abc2-ec42809edbfe tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Router c2ffed60-7ad5-4215-9e01-680707fb98d6 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:23:28.697837 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d634024c-b245-4d3e-abc2-ec42809edbfe tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:23:28.708280 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d4785306-2df6-4ac5-9b04-63190b074ab2 None None] Sync routers for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:23:28.712223 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d634024c-b245-4d3e-abc2-ec42809edbfe tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:28.721874 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d4785306-2df6-4ac5-9b04-63190b074ab2 None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:23:28.722800 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-d4785306-2df6-4ac5-9b04-63190b074ab2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:23:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:23:28.769652 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-d634024c-b245-4d3e-abc2-ec42809edbfe tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "PUT /v2.0/floatingips/1f2903d8-b12d-4d40-bb9f-84aa6c3e7266 HTTP/1.1" status: 200 len: 991 time: 1.6589603 May 24 11:23:28.770427 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:23:28.821459 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'status': 'completed', 'code': 200} May 24 11:23:28.935977 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83dc37a1-bd94-4ab6-a1a5-0518e1726fb7 req-cc12e471-088b-49f6-87ba-ccd66f6ad79f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:28.938618 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:28.939134 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-83dc37a1-bd94-4ab6-a1a5-0518e1726fb7 req-cc12e471-088b-49f6-87ba-ccd66f6ad79f service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 1098 time: 0.0896118 May 24 11:23:28.986230 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-d4785306-2df6-4ac5-9b04-63190b074ab2 None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:23:29.080239 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83dc37a1-bd94-4ab6-a1a5-0518e1726fb7 req-d7bd23b3-6495-4d22-a66e-c6220a0aa668 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:29.081715 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-83dc37a1-bd94-4ab6-a1a5-0518e1726fb7 req-d7bd23b3-6495-4d22-a66e-c6220a0aa668 service neutron] 10.222.0.93 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 917 time: 0.1364360 May 24 11:23:29.300068 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-83dc37a1-bd94-4ab6-a1a5-0518e1726fb7 req-ecf24f95-262f-4d0e-8fc0-69af39e1b9a3 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=a92e9868-2cc3-4d24-859b-18840b6fce56 HTTP/1.1" status: 200 len: 999 time: 0.2112799 May 24 11:23:29.344187 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-d4785306-2df6-4ac5-9b04-63190b074ab2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '1f2903d8-b12d-4d40-bb9f-84aa6c3e7266', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'floating_ip_address': '172.27.0.143', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'port_id': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 556, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'mac_address': 'fa:16:3e:48:62:19', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:23:13Z', 'updated_at': '2021-05-24T11:23:27Z', 'revision_number': 1, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:26Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:23:29.348378 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d4785306-2df6-4ac5-9b04-63190b074ab2 None None] Checking router: c2ffed60-7ad5-4215-9e01-680707fb98d6 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:23:29.356744 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d4785306-2df6-4ac5-9b04-63190b074ab2 None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:23:29.357401 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-d4785306-2df6-4ac5-9b04-63190b074ab2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:23:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:23:29.363125 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83dc37a1-bd94-4ab6-a1a5-0518e1726fb7 req-6f1b1fc2-07b0-44dd-a1a4-b77920948798 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:29.364620 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-83dc37a1-bd94-4ab6-a1a5-0518e1726fb7 req-6f1b1fc2-07b0-44dd-a1a4-b77920948798 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=12ff35e8-9af8-4339-95c9-c62c90d8c6fd HTTP/1.1" status: 200 len: 855 time: 0.0579495 May 24 11:23:29.442601 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83dc37a1-bd94-4ab6-a1a5-0518e1726fb7 req-3d2e3d0f-6f71-461d-8087-6cf3d2f748bb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:29.444054 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-83dc37a1-bd94-4ab6-a1a5-0518e1726fb7 req-3d2e3d0f-6f71-461d-8087-6cf3d2f748bb service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=7877e643-fe9e-4041-91fa-3a4f44279fd2&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0720975 May 24 11:23:29.541873 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-d4785306-2df6-4ac5-9b04-63190b074ab2 None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:23:29.580561 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-83dc37a1-bd94-4ab6-a1a5-0518e1726fb7 req-293034c6-fe63-4621-916b-f0d71f35d070 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1290920 May 24 11:23:29.722350 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-83dc37a1-bd94-4ab6-a1a5-0518e1726fb7 req-87da31ee-8a3c-4b9c-b59f-b80e0248526a service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1345551 May 24 11:23:29.895036 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-d4785306-2df6-4ac5-9b04-63190b074ab2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '1f2903d8-b12d-4d40-bb9f-84aa6c3e7266', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'floating_ip_address': '172.27.0.143', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'port_id': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 556, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'mac_address': 'fa:16:3e:48:62:19', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:23:13Z', 'updated_at': '2021-05-24T11:23:27Z', 'revision_number': 1, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:26Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:23:29.898972 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d4785306-2df6-4ac5-9b04-63190b074ab2 None None] The sync data for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi is: [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': '1f2903d8-b12d-4d40-bb9f-84aa6c3e7266', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'floating_ip_address': '172.27.0.143', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'port_id': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 556, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'mac_address': 'fa:16:3e:48:62:19', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:23:13Z', 'updated_at': '2021-05-24T11:23:27Z', 'revision_number': 1, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:26Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:23:30.098561 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dc3f4190-c345-42db-985b-4d2b6c92f526 None None] New status for floating IP 1f2903d8-b12d-4d40-bb9f-84aa6c3e7266: ACTIVE {{(pid=91821) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:23:32.807411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 60248) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:32.923515 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3eef98f3-0b69-448d-b4d3-91693302f6c3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:32.943494 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3eef98f3-0b69-448d-b4d3-91693302f6c3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:32.947196 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-3eef98f3-0b69-448d-b4d3-91693302f6c3 admin admin] Usage tracker for resource:security_group_rule and tenant:5a2525f4b2d743f18140d3beed47fa4c is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:32.950600 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-3eef98f3-0b69-448d-b4d3-91693302f6c3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:32.957423 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3eef98f3-0b69-448d-b4d3-91693302f6c3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:33.036158 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3eef98f3-0b69-448d-b4d3-91693302f6c3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.040848 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3eef98f3-0b69-448d-b4d3-91693302f6c3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.048571 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3ec168a0-5218-4a9c-a4cc-fb936ace2b45 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:33.087697 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3eef98f3-0b69-448d-b4d3-91693302f6c3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=586dfe0d-6f98-40a2-bd82-0043f100bf10,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:33.091355 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3eef98f3-0b69-448d-b4d3-91693302f6c3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:33.093653 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-3eef98f3-0b69-448d-b4d3-91693302f6c3 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=5a2525f4b2d743f18140d3beed47fa4c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2852433 May 24 11:23:33.097599 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 60254) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:33.158635 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e01ec0c2-1347-40eb-825e-9bf17975295e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.185466 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e01ec0c2-1347-40eb-825e-9bf17975295e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.241353 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e01ec0c2-1347-40eb-825e-9bf17975295e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.244871 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c533b884-2733-4003-a0c6-f02d300b6eac admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:33.260759 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e01ec0c2-1347-40eb-825e-9bf17975295e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=586dfe0d-6f98-40a2-bd82-0043f100bf10,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:33.264965 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e01ec0c2-1347-40eb-825e-9bf17975295e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:33.268501 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e01ec0c2-1347-40eb-825e-9bf17975295e admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.272586 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e01ec0c2-1347-40eb-825e-9bf17975295e admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/586dfe0d-6f98-40a2-bd82-0043f100bf10 HTTP/1.1" status: 204 len: 168 time: 0.1740324 May 24 11:23:33.554541 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 60264) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:33.572179 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-fe4687a0-c751-454f-863a-a6e8b5386c9a admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.587946 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-fe4687a0-c751-454f-863a-a6e8b5386c9a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:33.593323 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-fe4687a0-c751-454f-863a-a6e8b5386c9a admin admin] Usage tracker for resource:security_group_rule and tenant:a6693b10ad0540568c6ed89381e4a8a8 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:33.596602 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-fe4687a0-c751-454f-863a-a6e8b5386c9a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:33.602247 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-fe4687a0-c751-454f-863a-a6e8b5386c9a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:33.688562 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-fe4687a0-c751-454f-863a-a6e8b5386c9a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.692221 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-fe4687a0-c751-454f-863a-a6e8b5386c9a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.702961 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d036c90a-ef32-40b3-a0b7-a1e1feb27369 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:33.739904 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-fe4687a0-c751-454f-863a-a6e8b5386c9a admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=a6693b10ad0540568c6ed89381e4a8a8&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1842909 May 24 11:23:33.741572 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fe4687a0-c751-454f-863a-a6e8b5386c9a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e93ff452-7e00-40a3-b611-f264a10e1a16,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:33.742534 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 60268) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:33.744639 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-fe4687a0-c751-454f-863a-a6e8b5386c9a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:33.805245 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-668fe03a-6acb-40d8-9bb1-56b47cbf9974 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.830490 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-668fe03a-6acb-40d8-9bb1-56b47cbf9974 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.882754 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-668fe03a-6acb-40d8-9bb1-56b47cbf9974 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.887428 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0e503538-937b-483c-bf34-ba38d5f97b0c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:33.899165 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-668fe03a-6acb-40d8-9bb1-56b47cbf9974 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e93ff452-7e00-40a3-b611-f264a10e1a16,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:33.901529 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-668fe03a-6acb-40d8-9bb1-56b47cbf9974 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:33.904747 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-668fe03a-6acb-40d8-9bb1-56b47cbf9974 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:33.906568 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-668fe03a-6acb-40d8-9bb1-56b47cbf9974 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/e93ff452-7e00-40a3-b611-f264a10e1a16 HTTP/1.1" status: 204 len: 168 time: 0.1628766 May 24 11:23:35.907552 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 60332) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:36.028604 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e122ec75-d172-494a-9600-b9cae8bebbe5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:36.044590 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-e122ec75-d172-494a-9600-b9cae8bebbe5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:36.048763 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-e122ec75-d172-494a-9600-b9cae8bebbe5 admin admin] Usage tracker for resource:security_group_rule and tenant:ded697cddf1248f78c8f3196217b0380 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:36.051844 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-e122ec75-d172-494a-9600-b9cae8bebbe5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:36.059800 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-e122ec75-d172-494a-9600-b9cae8bebbe5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:36.143979 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e122ec75-d172-494a-9600-b9cae8bebbe5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:36.148113 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-e122ec75-d172-494a-9600-b9cae8bebbe5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:36.156943 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-4b74dfc1-937e-4208-9ad5-3ea2350b720a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:36.198704 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-e122ec75-d172-494a-9600-b9cae8bebbe5 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=ded697cddf1248f78c8f3196217b0380&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2900302 May 24 11:23:36.200014 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e122ec75-d172-494a-9600-b9cae8bebbe5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=3fa02dc2-1542-41cb-9913-05b0f9fb7073,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:36.201659 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 60338) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:36.202987 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e122ec75-d172-494a-9600-b9cae8bebbe5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:36.263452 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-325dac9c-29e2-45ad-8137-5d447039267f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:36.289554 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-325dac9c-29e2-45ad-8137-5d447039267f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:36.353083 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-325dac9c-29e2-45ad-8137-5d447039267f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:36.357605 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e44066e2-4467-4f3b-b358-36703fb96b03 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:36.369695 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-325dac9c-29e2-45ad-8137-5d447039267f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=3fa02dc2-1542-41cb-9913-05b0f9fb7073,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:36.372551 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-325dac9c-29e2-45ad-8137-5d447039267f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:36.384238 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-325dac9c-29e2-45ad-8137-5d447039267f admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:36.385379 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-325dac9c-29e2-45ad-8137-5d447039267f admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/3fa02dc2-1542-41cb-9913-05b0f9fb7073 HTTP/1.1" status: 204 len: 168 time: 0.1826098 May 24 11:23:36.689043 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 60348) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:36.707501 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e3b527fc-a57c-4c2a-a256-502bab9c819e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:36.724819 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e3b527fc-a57c-4c2a-a256-502bab9c819e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:36.728937 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e3b527fc-a57c-4c2a-a256-502bab9c819e admin admin] Usage tracker for resource:security_group_rule and tenant:b1b1609c231d4d2aaec7e0b1ae4edc80 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:36.732752 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e3b527fc-a57c-4c2a-a256-502bab9c819e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:36.738710 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e3b527fc-a57c-4c2a-a256-502bab9c819e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:36.825682 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e3b527fc-a57c-4c2a-a256-502bab9c819e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:36.829426 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e3b527fc-a57c-4c2a-a256-502bab9c819e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:36.836448 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e01ec0c2-1347-40eb-825e-9bf17975295e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:36.875970 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e3b527fc-a57c-4c2a-a256-502bab9c819e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=3f13a737-0756-438d-b16e-cd6260284c2c,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:36.880170 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e3b527fc-a57c-4c2a-a256-502bab9c819e admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=b1b1609c231d4d2aaec7e0b1ae4edc80&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1899741 May 24 11:23:36.881115 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e3b527fc-a57c-4c2a-a256-502bab9c819e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:36.882928 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 60352) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:36.944244 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-870969b0-bbce-4c83-84cf-719a0ca91b33 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:36.970403 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-870969b0-bbce-4c83-84cf-719a0ca91b33 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:37.020964 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-870969b0-bbce-4c83-84cf-719a0ca91b33 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:37.025568 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f815f17d-565e-47a3-9b43-685267af9168 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:37.038548 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-870969b0-bbce-4c83-84cf-719a0ca91b33 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=3f13a737-0756-438d-b16e-cd6260284c2c,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:37.041044 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-870969b0-bbce-4c83-84cf-719a0ca91b33 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:37.042254 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-870969b0-bbce-4c83-84cf-719a0ca91b33 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:37.043411 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-870969b0-bbce-4c83-84cf-719a0ca91b33 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/3f13a737-0756-438d-b16e-cd6260284c2c HTTP/1.1" status: 204 len: 168 time: 0.1595902 May 24 11:23:43.292442 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:23:49.411067 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 60486) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:49.542205 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:49.556973 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:49.560863 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Usage tracker for resource:security_group_rule and tenant:efc9fe408c04464a8dd2a106197f7627 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:49.563653 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:49.569521 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:49.654882 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:49.659749 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:49.666449 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d34c53e4-13fa-4305-a86a-fc3c108d7e70 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:49.708631 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=efc9fe408c04464a8dd2a106197f7627&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2964933 May 24 11:23:49.709580 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c14cd6f3-57dc-4748-b826-7c60e9491e1a,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:49.711372 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 60492) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:49.717844 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.718375 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.719211 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.719808 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.720491 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.721257 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.721744 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.722259 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.722828 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.723315 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.723824 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.724323 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.730486 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.064s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:49.778938 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:49.804237 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:49.853994 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:49.858097 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-3a9b903c-8fff-4e59-9d3b-7e4519b33767 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:49.871429 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c14cd6f3-57dc-4748-b826-7c60e9491e1a,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:49.884123 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.884486 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.884843 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.885227 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.885768 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.886018 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.886432 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.886640 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.886988 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.887336 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.887647 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.887985 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:49.891308 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:49.892508 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:49.893715 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-3d3f76ed-4a8f-44c4-849b-49402450f5bb admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/c14cd6f3-57dc-4748-b826-7c60e9491e1a HTTP/1.1" status: 204 len: 168 time: 0.1812117 May 24 11:23:50.184911 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 60500) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:50.209956 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a5169476-5f3d-44ee-b6b3-cc5031f8e5b4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:50.228492 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-a5169476-5f3d-44ee-b6b3-cc5031f8e5b4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:50.232358 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-a5169476-5f3d-44ee-b6b3-cc5031f8e5b4 admin admin] Usage tracker for resource:security_group_rule and tenant:ba90e50a50ac4846babed37eff807f66 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:50.235318 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-a5169476-5f3d-44ee-b6b3-cc5031f8e5b4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:50.241216 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-a5169476-5f3d-44ee-b6b3-cc5031f8e5b4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:50.349861 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a5169476-5f3d-44ee-b6b3-cc5031f8e5b4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:50.353733 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a5169476-5f3d-44ee-b6b3-cc5031f8e5b4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:50.361402 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ba6cedee-b0cd-475b-86ad-4150326ad012 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:50.402469 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-a5169476-5f3d-44ee-b6b3-cc5031f8e5b4 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=ba90e50a50ac4846babed37eff807f66&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2163148 May 24 11:23:50.403390 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a5169476-5f3d-44ee-b6b3-cc5031f8e5b4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=369f2b95-e9b1-438d-98eb-105157785397,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:50.405464 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 60504) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:23:50.408213 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a5169476-5f3d-44ee-b6b3-cc5031f8e5b4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:50.470288 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-41bea760-99db-465e-940c-730da2e8db3b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:50.496925 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-41bea760-99db-465e-940c-730da2e8db3b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:50.557038 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-41bea760-99db-465e-940c-730da2e8db3b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:50.562180 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e122ec75-d172-494a-9600-b9cae8bebbe5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:50.583264 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-41bea760-99db-465e-940c-730da2e8db3b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=369f2b95-e9b1-438d-98eb-105157785397,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:50.585891 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-41bea760-99db-465e-940c-730da2e8db3b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:50.587205 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-41bea760-99db-465e-940c-730da2e8db3b admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:50.588288 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-41bea760-99db-465e-940c-730da2e8db3b admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/369f2b95-e9b1-438d-98eb-105157785397 HTTP/1.1" status: 204 len: 168 time: 0.1814520 May 24 11:23:52.076291 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:53.285856 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-6056194f-0b97-4d06-816b-320387046c54 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/networks?tenant_id=cc4848b2adf14dc2980239ac51312aef&shared=False HTTP/1.1" status: 200 len: 213 time: 0.0298574 May 24 11:23:53.430415 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e17a9d86-48d1-4381-aec8-ef952ddd016d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:53.448284 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-3788b789-e62b-4d3c-ac9a-91ccb3ed56b2 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:53.449748 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-3788b789-e62b-4d3c-ac9a-91ccb3ed56b2 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 769 time: 0.1589525 May 24 11:23:53.466618 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-aa31f713-23fa-404f-a520-453f1d1ac00f tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/quotas/cc4848b2adf14dc2980239ac51312aef HTTP/1.1" status: 200 len: 375 time: 0.0103016 May 24 11:23:53.518464 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-8843c2c7-fee1-4f5e-a7cd-4636957a852c tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=cc4848b2adf14dc2980239ac51312aef&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0450041 May 24 11:23:53.714770 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-af8a8749-c2ff-40a2-b010-328967eca77f req-51c7e2a4-1c4f-4767-baaa-32878dbe2818 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 210 time: 0.0454824 May 24 11:23:54.826009 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-91a07c98-82f9-4c4f-afe3-34fef428af11 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.93 "GET /v2.0/networks?tenant_id=cc4848b2adf14dc2980239ac51312aef&shared=False HTTP/1.1" status: 200 len: 213 time: 0.0358002 May 24 11:23:54.958343 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-23ce4ad1-5bde-46c2-a917-b27742bb2639 req-b8069fcc-38dd-411c-957d-40047056b921 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 210 time: 0.1017854 May 24 11:23:55.063610 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-00709e78-b78a-4fa3-9909-6a2e20c5b123 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:55.065044 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-00709e78-b78a-4fa3-9909-6a2e20c5b123 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.93 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 769 time: 0.2326813 May 24 11:23:55.082848 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Request body: {'port': {'device_id': '53016597-9097-4d3c-a952-9046d711c443', 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a', 'admin_state_up': True, 'tenant_id': 'cc4848b2adf14dc2980239ac51312aef'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:23:55.221397 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:55.225209 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Usage tracker for resource:port and tenant:cc4848b2adf14dc2980239ac51312aef is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:55.227981 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Quota usage for port was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:55.233824 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:55.243926 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Made reservation on behalf of cc4848b2adf14dc2980239ac51312aef for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:23:55.247965 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:55.260960 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:55.278665 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:23:55.282563 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Usage tracker for resource:security_group_rule and tenant:cc4848b2adf14dc2980239ac51312aef is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:23:55.285405 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:23:55.291804 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:23:55.406009 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:55.411016 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:55.413910 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e3b527fc-a57c-4c2a-a256-502bab9c819e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:55.487271 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Pushing event updated for resources: {'SecurityGroup': ['ID=9dbac750-deb6-4a3f-b483-2a5515a37c05,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:55.492626 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:55.603075 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Allocated IP 192.168.233.161 (f5351ce6-b1a3-4ac8-a776-f21bf320429a/6328b963-5efa-47e7-8def-4502c360c671/944f3ce5-8a60-4b7b-a057-65babe74c6c0) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:23:55.751441 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:55.876453 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-0b9c5959-39b6-499b-a00e-0e0a272a08ba None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:55.901575 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:55.916979 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Transition to ACTIVE for port object 944f3ce5-8a60-4b7b-a057-65babe74c6c0 will not be triggered until provisioned by entity DHCP. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:23:55.940878 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:55.946364 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4579ebdd-a23e-4b77-ac5a-26d5b21d74e0 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:56.015844 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Network f5351ce6-b1a3-4ac8-a776-f21bf320429a is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:23:56.019015 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:23:56.022881 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:23:56.055417 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Pushing event updated for resources: {'Port': ['ID=944f3ce5-8a60-4b7b-a057-65babe74c6c0,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:56.077145 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.131s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:56.095897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:56.096536 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:56.215201 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 938 time: 1.1359997 May 24 11:23:56.226953 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Request body: {'port': {'device_id': '53016597-9097-4d3c-a952-9046d711c443', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:23:56.254178 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-a3d7978f-1ca8-4315-ae1d-678a2567d29d None None] Provisioning for port 944f3ce5-8a60-4b7b-a057-65babe74c6c0 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:23:56.258993 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-a3d7978f-1ca8-4315-ae1d-678a2567d29d None None] Provisioning complete for port 944f3ce5-8a60-4b7b-a057-65babe74c6c0 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:23:56.259400 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a3d7978f-1ca8-4315-ae1d-678a2567d29d None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:56.269690 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9bb610f5-a7ca-4487-98a7-1b75ef83ba1a req-71592109-641f-4bab-8fba-99662a03b4f5 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:56.270315 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-9bb610f5-a7ca-4487-98a7-1b75ef83ba1a req-71592109-641f-4bab-8fba-99662a03b4f5 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Synchronizing usage tracker for tenant:cc4848b2adf14dc2980239ac51312aef on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:23:56.318123 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-a3d7978f-1ca8-4315-ae1d-678a2567d29d None None] Port 944f3ce5-8a60-4b7b-a057-65babe74c6c0 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:23:56.336177 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-9bb610f5-a7ca-4487-98a7-1b75ef83ba1a req-71592109-641f-4bab-8fba-99662a03b4f5 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Unset dirty status for tenant:cc4848b2adf14dc2980239ac51312aef on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:23:56.337350 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9bb610f5-a7ca-4487-98a7-1b75ef83ba1a req-71592109-641f-4bab-8fba-99662a03b4f5 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 936 time: 0.2365913 May 24 11:23:56.391626 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:56.409372 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-9bb610f5-a7ca-4487-98a7-1b75ef83ba1a req-357c47ea-e338-492b-a187-8b441602d4a6 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Synchronizing usage tracker for tenant:cc4848b2adf14dc2980239ac51312aef on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:23:56.475297 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-9bb610f5-a7ca-4487-98a7-1b75ef83ba1a req-357c47ea-e338-492b-a187-8b441602d4a6 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Unset dirty status for tenant:cc4848b2adf14dc2980239ac51312aef on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:23:56.476441 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9bb610f5-a7ca-4487-98a7-1b75ef83ba1a req-357c47ea-e338-492b-a187-8b441602d4a6 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/security-groups?id=9dbac750-deb6-4a3f-b483-2a5515a37c05&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1328003 May 24 11:23:56.770186 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 944f3ce5-8a60-4b7b-a057-65babe74c6c0 {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:23:56.777951 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:56.792711 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Transition to ACTIVE for port object 944f3ce5-8a60-4b7b-a057-65babe74c6c0 will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:23:56.797163 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:56.801368 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:56.961682 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Pushing event updated for resources: {'Port': ['ID=944f3ce5-8a60-4b7b-a057-65babe74c6c0,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:56.964672 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.163s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:56.990366 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:57.049620 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Attempting to bind port 944f3ce5-8a60-4b7b-a057-65babe74c6c0 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:23:57.050371 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Attempting to bind port 944f3ce5-8a60-4b7b-a057-65babe74c6c0 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': '18940746-bb2a-4fb9-aa85-f52be81dada5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 504, 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:23:57.050830 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Attempting to bind port 944f3ce5-8a60-4b7b-a057-65babe74c6c0 on network f5351ce6-b1a3-4ac8-a776-f21bf320429a {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:23:57.058388 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Port 944f3ce5-8a60-4b7b-a057-65babe74c6c0 on network f5351ce6-b1a3-4ac8-a776-f21bf320429a not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:23:57.058701 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Attempting to bind port 944f3ce5-8a60-4b7b-a057-65babe74c6c0 on network f5351ce6-b1a3-4ac8-a776-f21bf320429a {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:23:57.065130 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 23, 52), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:23:57.065597 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Checking segment: {'id': '18940746-bb2a-4fb9-aa85-f52be81dada5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 504, 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:23:57.066112 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Bound using segment: {'id': '18940746-bb2a-4fb9-aa85-f52be81dada5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 504, 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:23:57.066793 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Bound port: 944f3ce5-8a60-4b7b-a057-65babe74c6c0, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '18940746-bb2a-4fb9-aa85-f52be81dada5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 504, 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:23:57.067492 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:57.159876 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] For port 944f3ce5-8a60-4b7b-a057-65babe74c6c0, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:23:57.162870 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] For port 944f3ce5-8a60-4b7b-a057-65babe74c6c0, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=944f3ce5-8a60-4b7b-a057-65babe74c6c0,segment=NetworkSegment(18940746-bb2a-4fb9-aa85-f52be81dada5),segment_id=18940746-bb2a-4fb9-aa85-f52be81dada5)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:23:57.167594 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:57.172472 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-b75cdd68-8033-489f-91cc-e080dd76b336 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:57.236365 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:57.237018 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:57.238489 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] 10.222.0.93 "PUT /v2.0/ports/944f3ce5-8a60-4b7b-a057-65babe74c6c0 HTTP/1.1" status: 200 len: 1098 time: 1.0156720 May 24 11:23:57.238999 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '53016597-9097-4d3c-a952-9046d711c443', 'tag': '944f3ce5-8a60-4b7b-a057-65babe74c6c0'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:23:57.263723 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Pushing event updated for resources: {'Port': ['ID=944f3ce5-8a60-4b7b-a057-65babe74c6c0,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:57.270692 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:57.331001 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '53016597-9097-4d3c-a952-9046d711c443', 'tag': '944f3ce5-8a60-4b7b-a057-65babe74c6c0', 'status': 'completed', 'code': 200} May 24 11:23:57.372679 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-e4addc84-9d72-41d3-8aa6-2c2f0f905098 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:57.374196 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-e4addc84-9d72-41d3-8aa6-2c2f0f905098 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=cc4848b2adf14dc2980239ac51312aef&device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 1101 time: 0.0804181 May 24 11:23:57.461421 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-9603d91f-dd8c-4b4c-aeab-16e8325d5217 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=192.168.233.161&port_id=944f3ce5-8a60-4b7b-a057-65babe74c6c0 HTTP/1.1" status: 200 len: 217 time: 0.0768695 May 24 11:23:57.530643 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-dafddc71-0fc5-4413-b52f-dba332c74479 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:57.532080 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-dafddc71-0fc5-4413-b52f-dba332c74479 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=6328b963-5efa-47e7-8def-4502c360c671 HTTP/1.1" status: 200 len: 843 time: 0.0634501 May 24 11:23:57.612829 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-f64c03f6-4a9e-4ec9-b092-13e8c6a7289e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:57.614317 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-f64c03f6-4a9e-4ec9-b092-13e8c6a7289e service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=f5351ce6-b1a3-4ac8-a776-f21bf320429a&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1178 time: 0.0753491 May 24 11:23:57.702776 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-527fd010-c5eb-4afd-9e88-f2916e73510d req-e1540a27-e7aa-4a4a-9d7f-de6ded657c26 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:57.704359 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-527fd010-c5eb-4afd-9e88-f2916e73510d req-e1540a27-e7aa-4a4a-9d7f-de6ded657c26 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 948 time: 0.0931597 May 24 11:23:57.753361 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-527fd010-c5eb-4afd-9e88-f2916e73510d req-9f8b3d8e-fdf5-4915-8d17-6048550b9c5a tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/security-groups?id=9dbac750-deb6-4a3f-b483-2a5515a37c05&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0429289 May 24 11:23:57.860326 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-ea7b52e6-4862-4fcb-a62d-fbd91bce7dc7 service neutron] 10.222.0.93 "GET /v2.0/networks/f5351ce6-b1a3-4ac8-a776-f21bf320429a?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2388802 May 24 11:23:58.009569 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0b01fece-79aa-473c-84ab-980ae8d904fe req-6316c01a-3028-438b-8020-9769a2787e2e service neutron] 10.222.0.93 "GET /v2.0/networks/f5351ce6-b1a3-4ac8-a776-f21bf320429a?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1423731 May 24 11:23:58.142460 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9c76c986-de7d-48b0-8a20-5c170c9440b5 req-a66a514e-abbc-480c-9e46-746c170f8521 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:58.143940 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9c76c986-de7d-48b0-8a20-5c170c9440b5 req-a66a514e-abbc-480c-9e46-746c170f8521 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=cc4848b2adf14dc2980239ac51312aef&device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 1101 time: 0.0801921 May 24 11:23:58.288543 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9c76c986-de7d-48b0-8a20-5c170c9440b5 req-355e5a73-d3a7-48c3-a996-71bf5e93198a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:58.290036 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9c76c986-de7d-48b0-8a20-5c170c9440b5 req-355e5a73-d3a7-48c3-a996-71bf5e93198a service neutron] 10.222.0.93 "GET /v2.0/networks?id=f5351ce6-b1a3-4ac8-a776-f21bf320429a HTTP/1.1" status: 200 len: 870 time: 0.1395495 May 24 11:23:58.365052 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9c76c986-de7d-48b0-8a20-5c170c9440b5 req-f25c354c-ea84-4d61-8893-4635d0c5ddf0 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=192.168.233.161&port_id=944f3ce5-8a60-4b7b-a057-65babe74c6c0 HTTP/1.1" status: 200 len: 217 time: 0.0671287 May 24 11:23:58.431681 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9c76c986-de7d-48b0-8a20-5c170c9440b5 req-e1579393-e9a7-46d7-a9f7-963c29dfd8ae service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:58.433157 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9c76c986-de7d-48b0-8a20-5c170c9440b5 req-e1579393-e9a7-46d7-a9f7-963c29dfd8ae service neutron] 10.222.0.93 "GET /v2.0/subnets?id=6328b963-5efa-47e7-8def-4502c360c671 HTTP/1.1" status: 200 len: 843 time: 0.0600505 May 24 11:23:58.517470 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9c76c986-de7d-48b0-8a20-5c170c9440b5 req-4375c76f-f777-41b6-a7e4-c20bb86b84a0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:58.518975 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9c76c986-de7d-48b0-8a20-5c170c9440b5 req-4375c76f-f777-41b6-a7e4-c20bb86b84a0 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=f5351ce6-b1a3-4ac8-a776-f21bf320429a&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1178 time: 0.0788157 May 24 11:23:58.664360 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9c76c986-de7d-48b0-8a20-5c170c9440b5 req-dba3e85d-196b-49f1-9c93-b0f325c14567 service neutron] 10.222.0.93 "GET /v2.0/networks/f5351ce6-b1a3-4ac8-a776-f21bf320429a?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1312878 May 24 11:23:58.815086 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9c76c986-de7d-48b0-8a20-5c170c9440b5 req-ae90b14a-b1c4-4d82-8c55-637a0a90e889 service neutron] 10.222.0.93 "GET /v2.0/networks/f5351ce6-b1a3-4ac8-a776-f21bf320429a?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1439404 May 24 11:23:58.935156 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:58.978489 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c94a52b2-6844-4a69-b1bc-59ca62d03df3 req-5f744f16-c05d-4ec6-a5d9-97bb6cb229b8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:23:58.980018 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c94a52b2-6844-4a69-b1bc-59ca62d03df3 req-5f744f16-c05d-4ec6-a5d9-97bb6cb229b8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 948 time: 0.0945349 May 24 11:23:59.032824 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c94a52b2-6844-4a69-b1bc-59ca62d03df3 req-41ccf8a6-75db-42b1-93ea-ca09b77f7d22 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/security-groups?id=9dbac750-deb6-4a3f-b483-2a5515a37c05&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0462351 May 24 11:23:59.068826 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 944f3ce5-8a60-4b7b-a057-65babe74c6c0 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:23:59.215725 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 944f3ce5-8a60-4b7b-a057-65babe74c6c0, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=944f3ce5-8a60-4b7b-a057-65babe74c6c0,segment=NetworkSegment(18940746-bb2a-4fb9-aa85-f52be81dada5),segment_id=18940746-bb2a-4fb9-aa85-f52be81dada5)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:23:59.230532 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '944f3ce5-8a60-4b7b-a057-65babe74c6c0', 'network_id': 'f5351ce6-b1a3-4ac8-a776-f21bf320429a', 'port_id': '944f3ce5-8a60-4b7b-a057-65babe74c6c0', 'mac_address': 'fa:16:3e:80:d5:b6', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 504, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '6328b963-5efa-47e7-8def-4502c360c671', 'ip_address': '192.168.233.161'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:23:59.297060 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:59.316711 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 944f3ce5-8a60-4b7b-a057-65babe74c6c0 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:23:59.337943 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 944f3ce5-8a60-4b7b-a057-65babe74c6c0, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=944f3ce5-8a60-4b7b-a057-65babe74c6c0,segment=NetworkSegment(18940746-bb2a-4fb9-aa85-f52be81dada5),segment_id=18940746-bb2a-4fb9-aa85-f52be81dada5)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:23:59.345244 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:23:59.348369 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-2fd12ac0-4083-45e8-a4ca-ee56694e5016 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:23:59.425994 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=944f3ce5-8a60-4b7b-a057-65babe74c6c0,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:23:59.434378 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:59.434748 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:59.435085 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:59.435416 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:59.435757 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:59.436080 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:59.436398 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:59.436717 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:59.437036 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:59.437388 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:59.437704 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:59.438021 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:23:59.442319 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:23:59.592232 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['944f3ce5-8a60-4b7b-a057-65babe74c6c0'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:24:00.238662 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-977ba75b-3507-4208-b645-1e47ea0bf712 req-24fe9170-ba5c-4586-95c9-c1bd536ae6d5 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:00.240158 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-977ba75b-3507-4208-b645-1e47ea0bf712 req-24fe9170-ba5c-4586-95c9-c1bd536ae6d5 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 949 time: 0.0899663 May 24 11:24:00.288589 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-977ba75b-3507-4208-b645-1e47ea0bf712 req-e7bab9a8-7d5e-4370-ac59-af44a8e96b96 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/security-groups?id=9dbac750-deb6-4a3f-b483-2a5515a37c05&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0425372 May 24 11:24:01.078694 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 944f3ce5-8a60-4b7b-a057-65babe74c6c0 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:24:01.145321 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 944f3ce5-8a60-4b7b-a057-65babe74c6c0 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:24:01.149724 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 944f3ce5-8a60-4b7b-a057-65babe74c6c0 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:24:01.150686 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:01.265459 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:01.292934 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '53016597-9097-4d3c-a952-9046d711c443', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '944f3ce5-8a60-4b7b-a057-65babe74c6c0'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:24:01.303606 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 944f3ce5-8a60-4b7b-a057-65babe74c6c0, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=944f3ce5-8a60-4b7b-a057-65babe74c6c0,segment=NetworkSegment(18940746-bb2a-4fb9-aa85-f52be81dada5),segment_id=18940746-bb2a-4fb9-aa85-f52be81dada5)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:24:01.308392 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:01.310706 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-a4c65b80-6844-4803-97b9-ebfb0540dca8 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:24:01.344754 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '53016597-9097-4d3c-a952-9046d711c443', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '944f3ce5-8a60-4b7b-a057-65babe74c6c0', 'code': 200} May 24 11:24:01.385579 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=944f3ce5-8a60-4b7b-a057-65babe74c6c0,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:24:01.388878 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:24:01.500913 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7bc8f040-0ab5-4751-a68d-5582d6462f37 req-5e51771c-e765-457b-94f5-f50bd01431b1 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:01.502397 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7bc8f040-0ab5-4751-a68d-5582d6462f37 req-5e51771c-e765-457b-94f5-f50bd01431b1 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 950 time: 0.0867751 May 24 11:24:01.597677 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7bc8f040-0ab5-4751-a68d-5582d6462f37 req-a8d1c471-602a-4128-bdc5-644b9c729d91 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/security-groups?id=9dbac750-deb6-4a3f-b483-2a5515a37c05&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0892596 May 24 11:24:01.622941 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-272a7f7c-72a5-4457-a117-19b0dc5ebb85 req-d0fbf682-c56d-4009-9def-4f2cc6e8f527 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:01.624499 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-272a7f7c-72a5-4457-a117-19b0dc5ebb85 req-d0fbf682-c56d-4009-9def-4f2cc6e8f527 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 1103 time: 0.1027482 May 24 11:24:01.667319 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-272a7f7c-72a5-4457-a117-19b0dc5ebb85 req-12bdbbd2-ca57-4adb-9731-dc370e9cf881 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=9dbac750-deb6-4a3f-b483-2a5515a37c05&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0357404 May 24 11:24:02.790921 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e159673-1120-4400-99f3-5b23e4764c8e req-72f015fc-c69c-4e7f-89ef-0a25ab092e62 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:02.792282 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1e159673-1120-4400-99f3-5b23e4764c8e req-72f015fc-c69c-4e7f-89ef-0a25ab092e62 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 950 time: 0.0807574 May 24 11:24:02.837184 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1e159673-1120-4400-99f3-5b23e4764c8e req-f0879816-1805-4945-a764-8a0ba51dc4d9 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/security-groups?id=9dbac750-deb6-4a3f-b483-2a5515a37c05&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0392866 May 24 11:24:03.281211 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-54871a5d-04ac-47a3-9176-32c5180fe8a4 req-afb8f34c-1e4e-4ee2-940d-4d3a0c9bb916 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:03.282645 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-54871a5d-04ac-47a3-9176-32c5180fe8a4 req-afb8f34c-1e4e-4ee2-940d-4d3a0c9bb916 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 950 time: 0.0798795 May 24 11:24:03.335114 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-54871a5d-04ac-47a3-9176-32c5180fe8a4 req-5d781f45-1e98-4d60-b698-0a32efcc93d2 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/security-groups?id=9dbac750-deb6-4a3f-b483-2a5515a37c05&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0428421 May 24 11:24:04.529409 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c6493dee-cb49-4907-832e-8223cf543425 req-8ceb8144-d061-4079-82f2-dc22e586b002 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:04.530896 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c6493dee-cb49-4907-832e-8223cf543425 req-8ceb8144-d061-4079-82f2-dc22e586b002 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 950 time: 0.0816455 May 24 11:24:04.583374 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c6493dee-cb49-4907-832e-8223cf543425 req-b9040483-0e9a-4cef-bcd7-44be3c0ad802 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/security-groups?id=9dbac750-deb6-4a3f-b483-2a5515a37c05&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0464506 May 24 11:24:04.605811 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-360ce0f7-7f74-4b73-88e2-e5a878014f29 req-4bdd1ba9-c58f-4486-8a1e-f383a7d5bc06 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:04.607307 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-360ce0f7-7f74-4b73-88e2-e5a878014f29 req-4bdd1ba9-c58f-4486-8a1e-f383a7d5bc06 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.93 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 950 time: 0.0935926 May 24 11:24:04.751948 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-360ce0f7-7f74-4b73-88e2-e5a878014f29 req-5ecbe35f-3f7b-487e-9b6e-97f93caa56ce tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Deleting port 944f3ce5-8a60-4b7b-a057-65babe74c6c0 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:24:04.752298 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-360ce0f7-7f74-4b73-88e2-e5a878014f29 req-5ecbe35f-3f7b-487e-9b6e-97f93caa56ce tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:04.974683 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-360ce0f7-7f74-4b73-88e2-e5a878014f29 req-5ecbe35f-3f7b-487e-9b6e-97f93caa56ce tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] For port 944f3ce5-8a60-4b7b-a057-65babe74c6c0, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=944f3ce5-8a60-4b7b-a057-65babe74c6c0,segment=NetworkSegment(18940746-bb2a-4fb9-aa85-f52be81dada5),segment_id=18940746-bb2a-4fb9-aa85-f52be81dada5)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:24:04.975041 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-360ce0f7-7f74-4b73-88e2-e5a878014f29 req-5ecbe35f-3f7b-487e-9b6e-97f93caa56ce tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:05.101723 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-360ce0f7-7f74-4b73-88e2-e5a878014f29 req-5ecbe35f-3f7b-487e-9b6e-97f93caa56ce tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Calling delete_port for 944f3ce5-8a60-4b7b-a057-65babe74c6c0 owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:24:05.243894 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-360ce0f7-7f74-4b73-88e2-e5a878014f29 req-5ecbe35f-3f7b-487e-9b6e-97f93caa56ce tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:05.249080 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4baff469-3df5-43fa-b42c-825725544fff tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:24:05.297519 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5ecbe35f-3f7b-487e-9b6e-97f93caa56ce tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Pushing event deleted for resources: {'Port': ['ID=944f3ce5-8a60-4b7b-a057-65babe74c6c0,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:24:05.301258 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5ecbe35f-3f7b-487e-9b6e-97f93caa56ce tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:24:05.380427 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-360ce0f7-7f74-4b73-88e2-e5a878014f29 req-5ecbe35f-3f7b-487e-9b6e-97f93caa56ce tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] No DVR routers for this DVR port 944f3ce5-8a60-4b7b-a057-65babe74c6c0 on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:24:05.382951 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-360ce0f7-7f74-4b73-88e2-e5a878014f29 req-5ecbe35f-3f7b-487e-9b6e-97f93caa56ce tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:05.386188 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '53016597-9097-4d3c-a952-9046d711c443', 'name': 'network-vif-deleted', 'tag': '944f3ce5-8a60-4b7b-a057-65babe74c6c0'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:24:05.403548 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-360ce0f7-7f74-4b73-88e2-e5a878014f29 req-5ecbe35f-3f7b-487e-9b6e-97f93caa56ce tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.93 "DELETE /v2.0/ports/944f3ce5-8a60-4b7b-a057-65babe74c6c0 HTTP/1.1" status: 204 len: 173 time: 0.7902088 May 24 11:24:05.445398 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '53016597-9097-4d3c-a952-9046d711c443', 'name': 'network-vif-deleted', 'tag': '944f3ce5-8a60-4b7b-a057-65babe74c6c0', 'status': 'completed', 'code': 200} May 24 11:24:05.747023 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-de9cfa11-dd4b-4ec3-874d-5f385161c8af req-5d50aed1-c0b8-4050-81cb-08da6b999590 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Synchronizing usage tracker for tenant:cc4848b2adf14dc2980239ac51312aef on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:24:05.761954 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-de9cfa11-dd4b-4ec3-874d-5f385161c8af req-5d50aed1-c0b8-4050-81cb-08da6b999590 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Unset dirty status for tenant:cc4848b2adf14dc2980239ac51312aef on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:24:05.763193 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-de9cfa11-dd4b-4ec3-874d-5f385161c8af req-5d50aed1-c0b8-4050-81cb-08da6b999590 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] 10.222.0.18 "GET /v2.0/ports?device_id=53016597-9097-4d3c-a952-9046d711c443 HTTP/1.1" status: 200 len: 210 time: 0.0598452 May 24 11:24:07.094287 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 944f3ce5-8a60-4b7b-a057-65babe74c6c0 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:24:07.129947 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 944f3ce5-8a60-4b7b-a057-65babe74c6c0 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:24:07.130328 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 944f3ce5-8a60-4b7b-a057-65babe74c6c0 not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:24:11.653660 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29af7941-bcc8-471e-9318-08043cb0b96a req-916395fe-e739-4b1e-9912-24308a8cf36a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:11.655155 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-29af7941-bcc8-471e-9318-08043cb0b96a req-916395fe-e739-4b1e-9912-24308a8cf36a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.1351459 May 24 11:24:11.680564 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 60888) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:24:11.701607 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-29af7941-bcc8-471e-9318-08043cb0b96a req-3fd2f527-3699-4833-9748-779535c1bd1c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0407009 May 24 11:24:11.886938 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-ab0641a0-42aa-427a-afaf-9a0af3f859f0 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=cc4848b2adf14dc2980239ac51312aef&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2050514 May 24 11:24:11.889504 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 60894) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:24:11.950824 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5d6107d5-263a-4a3b-8ce7-b4f241eb5e86 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:11.976215 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5d6107d5-263a-4a3b-8ce7-b4f241eb5e86 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:12.040268 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5d6107d5-263a-4a3b-8ce7-b4f241eb5e86 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:12.043808 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-85325d48-07ef-4710-96ad-0bfe451fa324 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:24:12.056034 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5d6107d5-263a-4a3b-8ce7-b4f241eb5e86 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9dbac750-deb6-4a3f-b483-2a5515a37c05,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:24:12.059156 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-5d6107d5-263a-4a3b-8ce7-b4f241eb5e86 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:24:12.068883 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5d6107d5-263a-4a3b-8ce7-b4f241eb5e86 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:12.070219 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-5d6107d5-263a-4a3b-8ce7-b4f241eb5e86 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/9dbac750-deb6-4a3f-b483-2a5515a37c05 HTTP/1.1" status: 204 len: 168 time: 0.1794863 May 24 11:24:12.916032 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-91980505-ce80-43f8-93d3-0bce12acc73a req-60d97de2-bd46-4044-b59a-470981a15bd0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:12.918362 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-91980505-ce80-43f8-93d3-0bce12acc73a req-60d97de2-bd46-4044-b59a-470981a15bd0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.1039693 May 24 11:24:12.979901 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-91980505-ce80-43f8-93d3-0bce12acc73a req-c20fce23-bd90-4b0b-af7e-6e48e8f8b3d6 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0558321 May 24 11:24:14.258531 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c977320b-e1ff-4811-b2d8-0077f80ef5ae req-55033fb6-5661-465e-aa4c-ebc9e123305b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:14.260016 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c977320b-e1ff-4811-b2d8-0077f80ef5ae req-55033fb6-5661-465e-aa4c-ebc9e123305b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.1719539 May 24 11:24:14.305993 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-c977320b-e1ff-4811-b2d8-0077f80ef5ae req-6951cfe7-652a-46bd-ae6d-5bc5fb6fffdb tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0401464 May 24 11:24:15.540053 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d8ed54db-a885-44aa-975d-0ede808dceeb req-a200c2d6-adac-4957-aad6-47c4f42ff454 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:15.548995 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d8ed54db-a885-44aa-975d-0ede808dceeb req-a200c2d6-adac-4957-aad6-47c4f42ff454 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0807531 May 24 11:24:15.590971 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d8ed54db-a885-44aa-975d-0ede808dceeb req-8ddae6a0-740f-4a10-8881-94faaef44ae9 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0401778 May 24 11:24:16.776878 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29c15a50-7145-4efd-8a2a-46ac857642b8 req-214ab1e4-0be6-4c71-9635-f7fb4e1ca730 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:16.778257 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-29c15a50-7145-4efd-8a2a-46ac857642b8 req-214ab1e4-0be6-4c71-9635-f7fb4e1ca730 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0800407 May 24 11:24:16.833083 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-29c15a50-7145-4efd-8a2a-46ac857642b8 req-a976b83f-d9e5-49d8-98dc-fc3a8a87fc00 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0458715 May 24 11:24:18.066640 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b9d867a-cfd1-45a9-9509-b9476d60738e req-eebef2d1-fa96-48cc-b4d6-31fe0ff69cdf tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:18.068101 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b9d867a-cfd1-45a9-9509-b9476d60738e req-eebef2d1-fa96-48cc-b4d6-31fe0ff69cdf tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.1124451 May 24 11:24:18.113296 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b9d867a-cfd1-45a9-9509-b9476d60738e req-7a300b96-8b9c-40d8-bca0-c89951aae2e0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0390205 May 24 11:24:19.303719 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-39ca7ef4-7836-4224-9b2f-dc7c66479f85 req-eb57825c-c976-48de-8c04-b61c7f62d09d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:19.305709 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-39ca7ef4-7836-4224-9b2f-dc7c66479f85 req-eb57825c-c976-48de-8c04-b61c7f62d09d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0804250 May 24 11:24:19.349477 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-39ca7ef4-7836-4224-9b2f-dc7c66479f85 req-c41026f9-cade-4a15-ac63-c53c653b349d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0381238 May 24 11:24:20.291549 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:24:20.578395 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c16786f6-da4d-4b4c-910e-e815c6629531 req-baea3bf3-269a-4cd1-ad50-cbb7ef446dfe tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:20.579857 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c16786f6-da4d-4b4c-910e-e815c6629531 req-baea3bf3-269a-4cd1-ad50-cbb7ef446dfe tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.1162193 May 24 11:24:20.629140 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c16786f6-da4d-4b4c-910e-e815c6629531 req-3038bae3-9fd1-49f0-96be-9d187385f9f3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0433636 May 24 11:24:20.939289 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-29de74b8-2c98-4ca9-af86-9f295fa8df7f req-4d0e1e36-9ea3-4958-98d7-bfd32a02411e service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0725887 May 24 11:24:21.074101 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29de74b8-2c98-4ca9-af86-9f295fa8df7f req-7b9ed769-f514-401d-872a-ea53293a6448 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:21.083281 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-29de74b8-2c98-4ca9-af86-9f295fa8df7f req-7b9ed769-f514-401d-872a-ea53293a6448 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 1098 time: 0.0785472 May 24 11:24:21.299073 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29de74b8-2c98-4ca9-af86-9f295fa8df7f req-545c9546-5b6c-4260-81de-e47d7d68088d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:21.300422 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-29de74b8-2c98-4ca9-af86-9f295fa8df7f req-545c9546-5b6c-4260-81de-e47d7d68088d service neutron] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 912 time: 0.1976604 May 24 11:24:21.405342 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-29de74b8-2c98-4ca9-af86-9f295fa8df7f req-a0fa80ce-f9e8-4234-86ca-3e2e7ec23c88 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=8ca4165e-4c84-4940-9148-3f5a77eaf348 HTTP/1.1" status: 200 len: 999 time: 0.0981846 May 24 11:24:21.470656 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29de74b8-2c98-4ca9-af86-9f295fa8df7f req-d827e775-ad03-48ea-b244-7e96fb4a9f79 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:21.472077 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-29de74b8-2c98-4ca9-af86-9f295fa8df7f req-d827e775-ad03-48ea-b244-7e96fb4a9f79 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.0609694 May 24 11:24:21.547154 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29de74b8-2c98-4ca9-af86-9f295fa8df7f req-8fbced02-e6e9-4ce9-8f68-a5cfd2f49773 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:21.548877 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-29de74b8-2c98-4ca9-af86-9f295fa8df7f req-8fbced02-e6e9-4ce9-8f68-a5cfd2f49773 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0700388 May 24 11:24:21.681495 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-29de74b8-2c98-4ca9-af86-9f295fa8df7f req-0631e212-5fc3-44dc-9ca7-efa4b4bfa9f1 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1256156 May 24 11:24:21.838832 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9f3e3352-b83d-469b-bde2-865fb7551cd5 req-b0e73ddc-18eb-4170-afe2-5b76d75d0935 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:21.840405 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9f3e3352-b83d-469b-bde2-865fb7551cd5 req-b0e73ddc-18eb-4170-afe2-5b76d75d0935 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0925574 May 24 11:24:21.888867 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9f3e3352-b83d-469b-bde2-865fb7551cd5 req-91f3094e-5413-435f-9dd6-1bfda05ee5c7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0416219 May 24 11:24:21.916258 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-29de74b8-2c98-4ca9-af86-9f295fa8df7f req-4c57c20d-3fb1-4465-a106-65c7118d4841 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2275913 May 24 11:24:22.061483 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:23.084627 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e1b5c57-c338-40ce-81d4-e408609091f8 req-207ec27a-2a18-4521-b630-d9bd47d63868 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:23.086700 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8e1b5c57-c338-40ce-81d4-e408609091f8 req-207ec27a-2a18-4521-b630-d9bd47d63868 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0812855 May 24 11:24:23.134191 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8e1b5c57-c338-40ce-81d4-e408609091f8 req-fc20ede6-5334-4886-ab97-ea8c2e13c939 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0416565 May 24 11:24:23.422378 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-24cba115-3070-4ac8-babc-9551b698252d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:24.387418 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d074a3da-f5cd-4874-bb07-5cf16a6713ed req-dac09300-6456-4d8e-93dd-1a56f04a7f5f tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:24.388893 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d074a3da-f5cd-4874-bb07-5cf16a6713ed req-dac09300-6456-4d8e-93dd-1a56f04a7f5f tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0910783 May 24 11:24:24.446944 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d074a3da-f5cd-4874-bb07-5cf16a6713ed req-60ad113b-e0e2-4732-868e-8644e5685979 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0507939 May 24 11:24:25.122477 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device a92e9868-2cc3-4d24-859b-18840b6fce56 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:24:25.262217 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=a92e9868-2cc3-4d24-859b-18840b6fce56,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:24:25.277255 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'port_id': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'mac_address': 'fa:16:3e:48:62:19', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 500, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.4'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:24:25.339655 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:25.364194 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: ACTIVE current_port_status: BUILD port_id a92e9868-2cc3-4d24-859b-18840b6fce56 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:24:25.382656 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=a92e9868-2cc3-4d24-859b-18840b6fce56,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:24:25.387600 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:25.389380 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:24:25.467032 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['a92e9868-2cc3-4d24-859b-18840b6fce56'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:24:25.509388 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=a92e9868-2cc3-4d24-859b-18840b6fce56,revision_number=6']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:24:25.518760 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:24:25.705027 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-03ed7ae6-afe1-497a-860c-5f6a0126eb38 req-1cfb6970-5248-4751-99b3-561e4e4edec3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:25.706451 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-03ed7ae6-afe1-497a-860c-5f6a0126eb38 req-1cfb6970-5248-4751-99b3-561e4e4edec3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 944 time: 0.0949588 May 24 11:24:25.755955 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:25.756432 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-03ed7ae6-afe1-497a-860c-5f6a0126eb38 req-c58272c7-d8de-4cbb-ad99-7b1a097ae763 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0433171 May 24 11:24:25.881024 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-6aeba7cc-38b5-4899-b25c-21c7ff4b94bd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:26.994555 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:27.003958 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-39d63a1f-a714-4856-8a68-73562d29ad1e req-56088b21-cf8b-420b-bf7f-6366b44029c1 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:27.008466 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-39d63a1f-a714-4856-8a68-73562d29ad1e req-56088b21-cf8b-420b-bf7f-6366b44029c1 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 944 time: 0.1162767 May 24 11:24:27.083924 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-39d63a1f-a714-4856-8a68-73562d29ad1e req-bf4e6a74-fda2-476c-96a5-1173e7f98b13 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0571907 May 24 11:24:27.131668 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device a92e9868-2cc3-4d24-859b-18840b6fce56 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:24:27.209481 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port a92e9868-2cc3-4d24-859b-18840b6fce56 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:24:27.210406 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:27.465748 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:27.517894 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:24:27.555982 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=a92e9868-2cc3-4d24-859b-18840b6fce56,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:24:27.561260 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:27.567160 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:24:27.666769 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'code': 200} May 24 11:24:27.670670 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=a92e9868-2cc3-4d24-859b-18840b6fce56,revision_number=7']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:24:27.677744 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:24:28.282153 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b2fd7de8-5577-45e7-92ad-4b3cad2d31a5 req-34fcd1e1-f906-480b-9365-745e68e00b3e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:28.284326 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b2fd7de8-5577-45e7-92ad-4b3cad2d31a5 req-34fcd1e1-f906-480b-9365-745e68e00b3e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0840397 May 24 11:24:28.338996 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b2fd7de8-5577-45e7-92ad-4b3cad2d31a5 req-6c7434e0-2f9a-4733-8283-c010f4e97ae1 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0485764 May 24 11:24:28.960991 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:29.556599 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8a885660-5fcc-47e2-823c-78e0e1422044 req-31aa9b40-8f97-475e-9fbb-7c5f512a6ab8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:29.558155 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8a885660-5fcc-47e2-823c-78e0e1422044 req-31aa9b40-8f97-475e-9fbb-7c5f512a6ab8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0954096 May 24 11:24:29.609028 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8a885660-5fcc-47e2-823c-78e0e1422044 req-71b99f63-7390-4835-94ae-458b8fb33bd0 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0418777 May 24 11:24:30.886940 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b8b5a60-4926-42e1-82b8-5624e9d1378e req-12334704-ccc9-48c1-9cec-7971e1ab9d38 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:30.888651 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2b8b5a60-4926-42e1-82b8-5624e9d1378e req-12334704-ccc9-48c1-9cec-7971e1ab9d38 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.1290617 May 24 11:24:30.944413 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2b8b5a60-4926-42e1-82b8-5624e9d1378e req-4b38e211-6610-4e02-8250-af670ee5eedd tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0469303 May 24 11:24:32.159558 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2567ce51-5c09-4e14-8406-adafc01fe37b req-9dfbd3d6-0f4e-4bb5-a4a2-7e7b580d6f4a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:32.166975 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2567ce51-5c09-4e14-8406-adafc01fe37b req-9dfbd3d6-0f4e-4bb5-a4a2-7e7b580d6f4a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0899661 May 24 11:24:32.216591 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-2567ce51-5c09-4e14-8406-adafc01fe37b req-6b37ffd1-8200-4f15-8001-8a63845f799b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0485599 May 24 11:24:33.491473 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f9735efe-ae70-4aa4-971e-d5c1e5cb2c88 req-cad1da92-84f2-4bf0-a94c-1e57ae209c8e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:33.492946 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f9735efe-ae70-4aa4-971e-d5c1e5cb2c88 req-cad1da92-84f2-4bf0-a94c-1e57ae209c8e tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0896723 May 24 11:24:33.559112 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f9735efe-ae70-4aa4-971e-d5c1e5cb2c88 req-04181425-bf87-47b7-a929-d66060c9aebd tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0582571 May 24 11:24:34.223974 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e0ad51a-e951-489e-9294-4620e9f30232 req-7d5ce1ae-1bda-4ae6-8359-3a49431a41ce service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:34.225644 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0e0ad51a-e951-489e-9294-4620e9f30232 req-7d5ce1ae-1bda-4ae6-8359-3a49431a41ce service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 1098 time: 0.0746806 May 24 11:24:34.356749 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e0ad51a-e951-489e-9294-4620e9f30232 req-04148043-0e60-492d-951b-89830b65dc57 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:34.358204 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0e0ad51a-e951-489e-9294-4620e9f30232 req-04148043-0e60-492d-951b-89830b65dc57 service neutron] 10.222.0.93 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 917 time: 0.1263421 May 24 11:24:34.463440 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0e0ad51a-e951-489e-9294-4620e9f30232 req-60f16668-d6aa-44cd-9209-4a735c4aad13 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=a92e9868-2cc3-4d24-859b-18840b6fce56 HTTP/1.1" status: 200 len: 1001 time: 0.0972800 May 24 11:24:34.526325 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e0ad51a-e951-489e-9294-4620e9f30232 req-80797400-517f-4ed7-bb2d-f7da8663fce5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:34.527641 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0e0ad51a-e951-489e-9294-4620e9f30232 req-80797400-517f-4ed7-bb2d-f7da8663fce5 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=12ff35e8-9af8-4339-95c9-c62c90d8c6fd HTTP/1.1" status: 200 len: 855 time: 0.0573690 May 24 11:24:34.609383 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e0ad51a-e951-489e-9294-4620e9f30232 req-91896070-7304-4163-a712-b3269ed7665e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:34.610891 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0e0ad51a-e951-489e-9294-4620e9f30232 req-91896070-7304-4163-a712-b3269ed7665e service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=7877e643-fe9e-4041-91fa-3a4f44279fd2&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0761859 May 24 11:24:34.806214 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0e0ad51a-e951-489e-9294-4620e9f30232 req-33e97151-5f8d-41a9-a2df-596b8ef96f3a service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1884279 May 24 11:24:34.840931 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-713dc628-442f-4d5e-be5b-d1f5e61f114e req-0ac473b5-5af5-49e0-beba-56f47ea95eb9 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:34.842409 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-713dc628-442f-4d5e-be5b-d1f5e61f114e req-0ac473b5-5af5-49e0-beba-56f47ea95eb9 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.1543221 May 24 11:24:34.963083 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-713dc628-442f-4d5e-be5b-d1f5e61f114e req-3ab33155-6205-496c-8edd-7effd7ff55ef tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0745714 May 24 11:24:34.986784 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0e0ad51a-e951-489e-9294-4620e9f30232 req-2c561a84-d9f0-4414-867e-166b071846ae service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1737938 May 24 11:24:36.158851 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b1718c39-84a7-41a4-b2cf-c1328045321e req-fdd3582a-aca9-4194-8e10-59e3b977fdd7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:36.160330 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b1718c39-84a7-41a4-b2cf-c1328045321e req-fdd3582a-aca9-4194-8e10-59e3b977fdd7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0803649 May 24 11:24:36.206312 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b1718c39-84a7-41a4-b2cf-c1328045321e req-c5711d43-63c6-4a4a-aa78-61b7f9c661ef tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0393784 May 24 11:24:37.415526 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-80de791f-15a6-4089-8d4c-baeb9084004e req-a53ff76f-7880-4ccd-bc37-39d9925d16d9 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:37.417006 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-80de791f-15a6-4089-8d4c-baeb9084004e req-a53ff76f-7880-4ccd-bc37-39d9925d16d9 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.0867047 May 24 11:24:37.462343 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-80de791f-15a6-4089-8d4c-baeb9084004e req-f5a6615d-aa8a-4f15-99cf-4a2c00876876 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0391026 May 24 11:24:38.907088 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-a5009de3-e8e2-4eed-abe2-259ad7855dc4 service neutron] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.0782809 May 24 11:24:38.943029 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fa68817f-32b4-438e-9e08-74fbc8aec2c3 req-066e034c-04d0-443e-8236-b8b36f31da2a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:38.944818 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fa68817f-32b4-438e-9e08-74fbc8aec2c3 req-066e034c-04d0-443e-8236-b8b36f31da2a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 945 time: 0.1178396 May 24 11:24:39.002197 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-fa68817f-32b4-438e-9e08-74fbc8aec2c3 req-3edf9c14-c945-419b-980a-0a508582b106 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0459709 May 24 11:24:39.172189 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device a92e9868-2cc3-4d24-859b-18840b6fce56 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:24:39.346530 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:39.374933 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:24:39.385342 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=a92e9868-2cc3-4d24-859b-18840b6fce56,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:24:39.396151 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:39.398098 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:24:39.428998 dv-cinder-784092-9-iscsi neutron-server[91821]: WARNING neutron.notifiers.nova [-] Nova event: {'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'name': 'network-vif-unplugged', 'status': 'failed', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'code': 422} returned with failed status May 24 11:24:39.472756 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=a92e9868-2cc3-4d24-859b-18840b6fce56,revision_number=8']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:24:39.476318 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:24:40.208205 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-69199636-514b-499b-8736-5992a744b4d5 req-7c5a9e49-faf5-4731-ae71-81b57f4932f8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:40.210917 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-69199636-514b-499b-8736-5992a744b4d5 req-7c5a9e49-faf5-4731-ae71-81b57f4932f8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.0926220 May 24 11:24:40.266522 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-69199636-514b-499b-8736-5992a744b4d5 req-1257d168-718a-4288-82c1-cd0c29af3845 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0500991 May 24 11:24:41.462835 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0c5852cf-e644-46da-8438-42ee1853724a req-cd1105b7-6178-4428-a8fc-3dc34d9681c7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:41.464321 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0c5852cf-e644-46da-8438-42ee1853724a req-cd1105b7-6178-4428-a8fc-3dc34d9681c7 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.0816312 May 24 11:24:41.509383 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0c5852cf-e644-46da-8438-42ee1853724a req-73c66841-d37e-49a6-8319-bd23bcc37ea1 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0391376 May 24 11:24:42.703245 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c50a9c70-b75b-43ed-bec3-4a81fe308778 req-875766d2-ea71-4b12-865a-2bcd5762b2ac tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:42.704865 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c50a9c70-b75b-43ed-bec3-4a81fe308778 req-875766d2-ea71-4b12-865a-2bcd5762b2ac tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.0814807 May 24 11:24:42.759977 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c50a9c70-b75b-43ed-bec3-4a81fe308778 req-074a18d7-a1df-4371-85bd-4ad84fb77e25 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0479333 May 24 11:24:42.948956 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-c288c08e-69ec-4a67-940e-07a133b6f67f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:42.950496 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-c288c08e-69ec-4a67-940e-07a133b6f67f service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed&tenant_id=05ec02ff35d2402e8d7bf064752e67ea HTTP/1.1" status: 200 len: 1096 time: 0.1818562 May 24 11:24:43.047945 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-327ea3aa-f9e2-4511-a894-4d9e73f82c56 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:43.049341 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-327ea3aa-f9e2-4511-a894-4d9e73f82c56 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 1096 time: 0.0738895 May 24 11:24:43.188599 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-0f012da2-172b-4601-9823-8482c3a89716 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:43.190081 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-0f012da2-172b-4601-9823-8482c3a89716 service neutron] 10.222.0.93 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 917 time: 0.1333055 May 24 11:24:43.293168 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-403123ef-bebb-499d-9671-c788893e73d7 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=a92e9868-2cc3-4d24-859b-18840b6fce56 HTTP/1.1" status: 200 len: 999 time: 0.0966465 May 24 11:24:43.356640 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-3724339f-0ade-44c1-985a-d40798d8dc69 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:43.358040 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-3724339f-0ade-44c1-985a-d40798d8dc69 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=12ff35e8-9af8-4339-95c9-c62c90d8c6fd HTTP/1.1" status: 200 len: 855 time: 0.0576589 May 24 11:24:43.436526 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-63b14366-d006-4fd4-982d-3d84b967e854 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:43.438047 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-63b14366-d006-4fd4-982d-3d84b967e854 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=7877e643-fe9e-4041-91fa-3a4f44279fd2&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0737679 May 24 11:24:43.573931 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-8ae19d1f-07fe-4d1c-ada1-5dc6dd42687c service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1295450 May 24 11:24:43.711980 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-769d78b6-a98c-4c2b-8e22-2c3d6479a9be req-aa498730-ff54-4b1a-8232-07705991231c service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1311526 May 24 11:24:44.001844 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e0fd5ea-b4e1-4824-88bd-dafc2f7c0885 req-f26f83c4-367f-44c9-80fc-f46da05f6649 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:44.004516 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6e0fd5ea-b4e1-4824-88bd-dafc2f7c0885 req-f26f83c4-367f-44c9-80fc-f46da05f6649 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.1282835 May 24 11:24:44.081838 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6e0fd5ea-b4e1-4824-88bd-dafc2f7c0885 req-6d3d2a54-aed5-48a3-aafd-b9a37d0d4db4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0586643 May 24 11:24:45.304977 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f32cb3c0-9dd0-407f-9fec-1c7cebd38512 req-bb2b5baa-1449-4323-b27a-73f8bc0745a9 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:45.306561 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f32cb3c0-9dd0-407f-9fec-1c7cebd38512 req-bb2b5baa-1449-4323-b27a-73f8bc0745a9 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.0942147 May 24 11:24:45.357388 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f32cb3c0-9dd0-407f-9fec-1c7cebd38512 req-bc7e1a0e-05e0-4017-a0e8-38af351bd846 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0431492 May 24 11:24:46.709740 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-474d0b81-b920-4c81-b61e-c36fbb075851 req-e49bcec2-67e2-4398-a5bd-7381991c1aa8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:46.712343 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-474d0b81-b920-4c81-b61e-c36fbb075851 req-e49bcec2-67e2-4398-a5bd-7381991c1aa8 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.1381214 May 24 11:24:46.792018 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-474d0b81-b920-4c81-b61e-c36fbb075851 req-05da8867-1040-438c-b50e-c526c30e6f58 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0737524 May 24 11:24:46.863182 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-928269d3-8700-4e93-b924-a000be116326 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-262467324&fields=id&tenant_id=9a8d61f79bbc496ca30e390fd9b5839e HTTP/1.1" status: 200 len: 267 time: 0.0585880 May 24 11:24:47.077564 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-ae53f43b-f313-4750-ae77-b6b6d3666de8 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:47.078979 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-ae53f43b-f313-4750-ae77-b6b6d3666de8 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 811 time: 0.2047427 May 24 11:24:47.095884 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-f36d9d44-1e84-405f-b9be-49519c44407b tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/quotas/9a8d61f79bbc496ca30e390fd9b5839e HTTP/1.1" status: 200 len: 375 time: 0.0101326 May 24 11:24:47.223473 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-b04f964c-31ad-4590-82a2-2dfd2a963556 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&fields=id HTTP/1.1" status: 200 len: 440 time: 0.1186078 May 24 11:24:47.428973 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9049361b-0b6b-4d76-8122-82ca1a19a987 service neutron] 10.222.0.18 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1800783 May 24 11:24:47.622346 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-86139f09-5ce4-458c-adb9-bdfb6d9edfbf service neutron] 10.222.0.18 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1836236 May 24 11:24:48.118531 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5a595dd5-8cd8-49f0-a435-93ad943840ff req-19b16233-f298-44b4-a492-299c5cd5bc64 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:48.128725 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5a595dd5-8cd8-49f0-a435-93ad943840ff req-19b16233-f298-44b4-a492-299c5cd5bc64 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.1175334 May 24 11:24:48.179649 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5a595dd5-8cd8-49f0-a435-93ad943840ff req-1c5724eb-e84f-4715-ba9d-8cfc53e822c5 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0496387 May 24 11:24:48.254391 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b80d65a-8222-42ef-8b31-327b4eb2be7e req-32094573-4a7f-49bc-aebe-03554bdc4889 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 210 time: 0.0871720 May 24 11:24:49.499422 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cb1475e5-cb4b-4c0c-b4bb-fa970e3e3a8d req-8badafb5-f89d-4c36-8601-0dc166250ead tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 210 time: 0.0529170 May 24 11:24:49.534993 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d895c05-76f2-4bbb-ae7f-78740fb4854a req-b82022cb-0113-4025-a616-d4532f46f0e4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:49.537470 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1d895c05-76f2-4bbb-ae7f-78740fb4854a req-b82022cb-0113-4025-a616-d4532f46f0e4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.1607733 May 24 11:24:49.595684 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-1d895c05-76f2-4bbb-ae7f-78740fb4854a req-27abfc64-04e0-41f2-a6db-f408c7df6ed6 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0482094 May 24 11:24:50.905633 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-055feadf-0e20-4509-9798-11b4c0d152d3 req-a04ed78f-b47d-41f0-a4c6-43f272280d36 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 210 time: 0.2201123 May 24 11:24:50.906343 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6fb76d41-9bc7-42a8-861e-64d451a0e438 req-fcae11a2-d720-4873-8eea-d5b0f9d9c980 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:50.909396 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6fb76d41-9bc7-42a8-861e-64d451a0e438 req-fcae11a2-d720-4873-8eea-d5b0f9d9c980 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.1461825 May 24 11:24:50.976741 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6fb76d41-9bc7-42a8-861e-64d451a0e438 req-30ecef79-1aee-4041-8982-b792f81dad00 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0583360 May 24 11:24:52.100813 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-91515501-6fc9-4069-90a6-ec04e4d7af77 req-17c0c062-c29b-4b68-bca0-8dfe3e3adbde tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 210 time: 0.0591893 May 24 11:24:52.101796 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:52.263520 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-436aa677-be3f-404e-acd4-c8f934358f94 req-fe7ed7cc-92a9-43d3-b5b4-e9883a85cf2d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:52.265679 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-436aa677-be3f-404e-acd4-c8f934358f94 req-fe7ed7cc-92a9-43d3-b5b4-e9883a85cf2d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.1290741 May 24 11:24:52.336065 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-436aa677-be3f-404e-acd4-c8f934358f94 req-83a44642-eb1c-4228-ac3b-d558efcfe44f tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0614221 May 24 11:24:53.294376 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f52475b3-38e2-403a-a26d-5e5246893281 req-e503f27a-d4e2-483b-b211-aa8c0abccd54 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 210 time: 0.0508683 May 24 11:24:53.429850 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-09770307-3023-4271-9f17-b379366807cb None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:53.586742 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-488211c6-2aa1-47f4-b2cb-fc25559f7026 req-51057c42-64c0-4dda-8af6-e6275f9a4939 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:53.588360 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-488211c6-2aa1-47f4-b2cb-fc25559f7026 req-51057c42-64c0-4dda-8af6-e6275f9a4939 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.0930972 May 24 11:24:53.641300 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-488211c6-2aa1-47f4-b2cb-fc25559f7026 req-8d06502f-bafe-4b39-953e-0a90b06367d9 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0453506 May 24 11:24:54.496487 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f3fa151c-196f-4052-9125-9ab66fa514bd req-28b039f1-7486-4d5b-9932-97107bc72f63 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 210 time: 0.0710702 May 24 11:24:54.969665 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ebefe28a-3ced-4d29-b2ff-f81ef0ca7300 req-d2ee3472-39a7-4ee0-9ffa-c77e598266e9 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:54.971815 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ebefe28a-3ced-4d29-b2ff-f81ef0ca7300 req-d2ee3472-39a7-4ee0-9ffa-c77e598266e9 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.2024624 May 24 11:24:55.032473 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ebefe28a-3ced-4d29-b2ff-f81ef0ca7300 req-d367edc9-b696-462f-92bb-96ad2ddc09bf tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0538821 May 24 11:24:55.684631 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-08a6eb63-f275-4f1c-a6bb-afce522d9236 req-6e48c6d5-5f9a-4244-a9e0-e6872af911a8 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 210 time: 0.0535109 May 24 11:24:55.762705 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:55.883570 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3e7c3c4b-f7e7-423c-ae4b-bfd882217e6a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:56.215029 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-48c969b8-e756-454a-be11-0d673ad36b56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:56.216513 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-48c969b8-e756-454a-be11-0d673ad36b56 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 811 time: 0.1908014 May 24 11:24:56.305422 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-0e6300c7-72a2-41b0-a2ed-04652ab8e65f tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=9a8d61f79bbc496ca30e390fd9b5839e HTTP/1.1" status: 200 len: 379 time: 0.0701370 May 24 11:24:56.315973 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'port': {'device_id': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'admin_state_up': True, 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'security_groups': ['5928fed2-3207-44a0-91e4-485200bd1a27']}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:24:56.348193 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-629bb0a0-d900-4206-8dcd-2ef55b54782f req-cc76d518-1606-4dac-9a49-07888ef62dbc tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:56.349793 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-629bb0a0-d900-4206-8dcd-2ef55b54782f req-cc76d518-1606-4dac-9a49-07888ef62dbc tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.1274276 May 24 11:24:56.408669 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-629bb0a0-d900-4206-8dcd-2ef55b54782f req-5f5f25b6-65ad-4caf-9aca-0a0a809e88af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0492897 May 24 11:24:56.489026 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:24:56.501543 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource port. Total usage: 4; quota limit: 500; headroom:496 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:24:56.518349 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Made reservation on behalf of 9a8d61f79bbc496ca30e390fd9b5839e for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:24:56.524337 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:56.783131 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Allocated IP 172.20.0.3 (be65d1af-351e-4e5b-8c19-1b04ff5cfc29/994391be-baae-4f60-9ef9-bf02d19d3477/f7c7da3d-a1d5-44d2-a684-ba1255d063df) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:24:56.912155 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bf79d45c-80fb-4656-9ebc-80d418d8602b req-1a07207e-14c0-4d37-b98a-03a0498e4de8 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 210 time: 0.0946825 May 24 11:24:56.992433 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:57.118722 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:57.134645 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Transition to ACTIVE for port object f7c7da3d-a1d5-44d2-a684-ba1255d063df will not be triggered until provisioned by entity DHCP. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:24:57.161192 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:57.167094 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-36620657-ffbb-4318-9bf5-88af723c5742 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:24:57.232212 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:24:57.235325 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:24:57.240718 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:24:57.278444 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'Port': ['ID=f7c7da3d-a1d5-44d2-a684-ba1255d063df,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:24:57.294667 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:24:57.296802 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:57.297198 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:57.297562 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:57.297924 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:57.298271 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:57.298621 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:57.298972 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:57.299319 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:57.299666 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:57.300007 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:57.300357 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:57.301509 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:57.310790 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.144s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:24:57.328216 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:57.328837 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:57.355711 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.0451579 May 24 11:24:57.368790 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Request body: {'port': {'device_id': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:24:57.489037 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:57.514696 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-658d4bbe-2083-4e4e-af4e-81da9deac41f None None] Provisioning for port f7c7da3d-a1d5-44d2-a684-ba1255d063df completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:24:57.520431 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-658d4bbe-2083-4e4e-af4e-81da9deac41f None None] Provisioning complete for port f7c7da3d-a1d5-44d2-a684-ba1255d063df triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:24:57.520798 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-658d4bbe-2083-4e4e-af4e-81da9deac41f None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:57.584808 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-658d4bbe-2083-4e4e-af4e-81da9deac41f None None] Port f7c7da3d-a1d5-44d2-a684-ba1255d063df cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:24:57.666390 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d1039f7-7855-478e-af23-5a2310110ecf req-edbc1fc6-0ddc-4304-a178-37aeeb8ac8c4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:57.668367 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6d1039f7-7855-478e-af23-5a2310110ecf req-edbc1fc6-0ddc-4304-a178-37aeeb8ac8c4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.0847461 May 24 11:24:57.716256 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6d1039f7-7855-478e-af23-5a2310110ecf req-e48f6cc7-974e-4321-8de0-8a6c9f4ca826 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0409033 May 24 11:24:57.875765 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id f7c7da3d-a1d5-44d2-a684-ba1255d063df {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:24:57.883441 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:57.898002 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Transition to ACTIVE for port object f7c7da3d-a1d5-44d2-a684-ba1255d063df will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:24:57.901647 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:57.906110 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e5adbcda-43bd-4ec7-bab1-e6df1d4d587e service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:24:58.028197 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Pushing event updated for resources: {'Port': ['ID=f7c7da3d-a1d5-44d2-a684-ba1255d063df,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:24:58.142512 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.236s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:24:58.309785 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Attempting to bind port f7c7da3d-a1d5-44d2-a684-ba1255d063df on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:24:58.310039 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Attempting to bind port f7c7da3d-a1d5-44d2-a684-ba1255d063df by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:24:58.310369 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Attempting to bind port f7c7da3d-a1d5-44d2-a684-ba1255d063df on network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:24:58.322992 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf5823ed-fbee-4d34-87ed-463895ef8be2 req-a6652291-8692-4d66-9fb1-29e4ca3b72d5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:58.323687 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-cf5823ed-fbee-4d34-87ed-463895ef8be2 req-a6652291-8692-4d66-9fb1-29e4ca3b72d5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Synchronizing usage tracker for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:24:58.326311 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Port f7c7da3d-a1d5-44d2-a684-ba1255d063df on network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:24:58.326637 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Attempting to bind port f7c7da3d-a1d5-44d2-a684-ba1255d063df on network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:24:58.335697 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 24, 52), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:24:58.336028 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Checking segment: {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:24:58.336415 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Bound using segment: {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:24:58.336845 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Bound port: f7c7da3d-a1d5-44d2-a684-ba1255d063df, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '2f7d3253-3331-43a2-800a-8858592fd924', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 503, 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:24:58.337393 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:58.387316 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-cf5823ed-fbee-4d34-87ed-463895ef8be2 req-a6652291-8692-4d66-9fb1-29e4ca3b72d5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Unset dirty status for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:24:58.388458 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cf5823ed-fbee-4d34-87ed-463895ef8be2 req-a6652291-8692-4d66-9fb1-29e4ca3b72d5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 943 time: 0.2481658 May 24 11:24:58.468278 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cf5823ed-fbee-4d34-87ed-463895ef8be2 req-e6117d91-8cd3-4528-9d30-f359012ec87e tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0723453 May 24 11:24:58.474790 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] For port f7c7da3d-a1d5-44d2-a684-ba1255d063df, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:24:58.478095 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] For port f7c7da3d-a1d5-44d2-a684-ba1255d063df, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=f7c7da3d-a1d5-44d2-a684-ba1255d063df,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:24:58.483784 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:58.487802 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-7c14a539-64ec-4b2f-999c-194c247731db tempest-AttachVolumeNegativeTest-829647707 tempest-AttachVolumeNegativeTest-829647707-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:24:58.549681 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:58.551250 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:58.553429 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] 10.222.0.93 "PUT /v2.0/ports/f7c7da3d-a1d5-44d2-a684-ba1255d063df HTTP/1.1" status: 200 len: 1093 time: 1.1901252 May 24 11:24:58.554152 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'tag': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:24:58.589284 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Pushing event updated for resources: {'Port': ['ID=f7c7da3d-a1d5-44d2-a684-ba1255d063df,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:24:58.594504 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:24:58.643070 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'tag': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df', 'status': 'completed', 'code': 200} May 24 11:24:58.695313 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-27dfab57-a0a6-40db-bdc0-60514a3c9099 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:58.699528 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-27dfab57-a0a6-40db-bdc0-60514a3c9099 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 1096 time: 0.0924587 May 24 11:24:58.774049 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-0411fa5f-8453-43a9-88d1-36224217aa12 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=f7c7da3d-a1d5-44d2-a684-ba1255d063df HTTP/1.1" status: 200 len: 217 time: 0.0672629 May 24 11:24:58.894523 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-395cf7db-fb4f-44a4-aadc-4b62b0d06514 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:58.900128 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-395cf7db-fb4f-44a4-aadc-4b62b0d06514 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.1164789 May 24 11:24:58.938616 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:59.055049 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40da5228-ca0a-4b2c-bbb1-30134eaec679 req-011cf40d-c187-491b-9946-079283d9973b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:59.057169 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-40da5228-ca0a-4b2c-bbb1-30134eaec679 req-011cf40d-c187-491b-9946-079283d9973b tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 943 time: 0.2133572 May 24 11:24:59.101984 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-cb2ad1c8-7aec-4f70-9fd4-2872d916beb5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:59.103473 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-cb2ad1c8-7aec-4f70-9fd4-2872d916beb5 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1899810 May 24 11:24:59.134120 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-40da5228-ca0a-4b2c-bbb1-30134eaec679 req-fa184c0c-8e2b-453c-8017-878b2f1b6157 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0710537 May 24 11:24:59.269331 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-ee083003-c0b0-4763-b2a0-c3c25d3b8a8c service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1587789 May 24 11:24:59.276482 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device a92e9868-2cc3-4d24-859b-18840b6fce56 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:24:59.436149 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932c59e5-6b5c-4e48-a822-2f3d36c7ce0e req-c8b12ad0-7be2-492a-9ae1-154e56f40e3d service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1597419 May 24 11:24:59.454728 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=a92e9868-2cc3-4d24-859b-18840b6fce56,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:24:59.471710 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'port_id': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'mac_address': 'fa:16:3e:48:62:19', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 500, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.4'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:24:59.543949 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:59.572635 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id a92e9868-2cc3-4d24-859b-18840b6fce56 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:24:59.579279 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28735619-b850-4e74-8b61-ae0269cde305 req-5dca9542-e6bf-4c2a-bfaf-cf5d72ac8164 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:59.581430 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-28735619-b850-4e74-8b61-ae0269cde305 req-5dca9542-e6bf-4c2a-bfaf-cf5d72ac8164 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 1096 time: 0.0887420 May 24 11:24:59.596797 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=a92e9868-2cc3-4d24-859b-18840b6fce56,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:24:59.602646 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:24:59.605937 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:24:59.792493 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=a92e9868-2cc3-4d24-859b-18840b6fce56,revision_number=9']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:24:59.805689 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28735619-b850-4e74-8b61-ae0269cde305 req-5e54a607-1d5b-4af5-ab82-d977cd06f70f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:59.807595 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-28735619-b850-4e74-8b61-ae0269cde305 req-5e54a607-1d5b-4af5-ab82-d977cd06f70f service neutron] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 912 time: 0.2201793 May 24 11:24:59.807865 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:59.808405 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:59.808744 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:59.809072 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:59.809417 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:59.809739 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:59.810056 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:59.810373 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:59.810702 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:59.811017 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:59.811318 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:59.811629 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:24:59.815616 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.210s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:24:59.825422 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fb785195-eca5-4664-b3fa-029fb6578a46 req-e31b188c-954b-40af-bf9e-1ab0bb61bd41 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:59.827705 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-fb785195-eca5-4664-b3fa-029fb6578a46 req-e31b188c-954b-40af-bf9e-1ab0bb61bd41 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 943 time: 0.1845024 May 24 11:24:59.867049 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['a92e9868-2cc3-4d24-859b-18840b6fce56'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:24:59.890404 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-28735619-b850-4e74-8b61-ae0269cde305 req-be134df3-53b2-427c-b98c-3baa175ec269 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=f7c7da3d-a1d5-44d2-a684-ba1255d063df HTTP/1.1" status: 200 len: 217 time: 0.0725312 May 24 11:24:59.894567 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-fb785195-eca5-4664-b3fa-029fb6578a46 req-2147a6a9-4408-451e-9757-778cee3b6d77 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0588992 May 24 11:24:59.958754 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28735619-b850-4e74-8b61-ae0269cde305 req-383201eb-0679-4fb8-9824-ba0f6fc8cf4b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:24:59.960388 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-28735619-b850-4e74-8b61-ae0269cde305 req-383201eb-0679-4fb8-9824-ba0f6fc8cf4b service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.0631711 May 24 11:25:00.043397 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28735619-b850-4e74-8b61-ae0269cde305 req-09267008-c9e7-4b1a-9ce5-73413ba2a295 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:00.045197 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-28735619-b850-4e74-8b61-ae0269cde305 req-09267008-c9e7-4b1a-9ce5-73413ba2a295 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0782983 May 24 11:25:00.328903 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-28735619-b850-4e74-8b61-ae0269cde305 req-5f965507-667f-4264-b77f-fc5aa82a6002 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2771749 May 24 11:25:00.452187 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70ace76a-e211-462a-8d63-86f60fe45c2b req-316ff46c-0b36-44ac-8938-6092a2f07e50 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:00.453671 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-70ace76a-e211-462a-8d63-86f60fe45c2b req-316ff46c-0b36-44ac-8938-6092a2f07e50 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 944 time: 0.1855898 May 24 11:25:00.513992 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-28735619-b850-4e74-8b61-ae0269cde305 req-bc893028-57e3-4d53-918b-49b146ebb6a5 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1776361 May 24 11:25:00.524013 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-70ace76a-e211-462a-8d63-86f60fe45c2b req-6bd3a5a6-4528-42da-b529-7b0e1461e726 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0597899 May 24 11:25:01.102959 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1d4e68c-4fd6-4c6b-8a9f-d4a30f6e9f92 req-e809a621-0dc2-4be6-b7ff-7952b4144845 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:01.104502 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d1d4e68c-4fd6-4c6b-8a9f-d4a30f6e9f92 req-e809a621-0dc2-4be6-b7ff-7952b4144845 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 943 time: 0.0897343 May 24 11:25:01.170753 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d1d4e68c-4fd6-4c6b-8a9f-d4a30f6e9f92 req-303397dd-891e-484e-b996-b401644e4205 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0585780 May 24 11:25:01.296260 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device a92e9868-2cc3-4d24-859b-18840b6fce56 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:25:01.374968 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port a92e9868-2cc3-4d24-859b-18840b6fce56 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:25:01.375510 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:01.547529 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:01.582667 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:25:01.594035 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=a92e9868-2cc3-4d24-859b-18840b6fce56,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:25:01.601623 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:01.603930 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:01.648121 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'code': 200} May 24 11:25:01.687934 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=a92e9868-2cc3-4d24-859b-18840b6fce56,revision_number=10']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:01.692202 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:01.793436 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75528f18-05ee-4b75-80b6-2b7872634515 req-3cf9cbc5-820d-4076-8913-4442ad9e2d2f tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:01.795308 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-75528f18-05ee-4b75-80b6-2b7872634515 req-3cf9cbc5-820d-4076-8913-4442ad9e2d2f tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 946 time: 0.0835781 May 24 11:25:01.894661 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-75528f18-05ee-4b75-80b6-2b7872634515 req-5da3c4a6-c734-49b1-93a0-d7cfd4c7afb4 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0929790 May 24 11:25:01.915554 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8a9517a7-a113-45ff-b8c9-99c6f330bdd9 req-d782c71c-8b21-4706-bfda-65b14c27764e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:01.917020 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8a9517a7-a113-45ff-b8c9-99c6f330bdd9 req-d782c71c-8b21-4706-bfda-65b14c27764e service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 1099 time: 0.0962987 May 24 11:25:01.961315 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-8a9517a7-a113-45ff-b8c9-99c6f330bdd9 req-96a37663-9f8a-46eb-9bd0-85887d39ffad service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0369513 May 24 11:25:02.357970 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-af7ff48c-0669-4b4a-9903-da57ad6dfbe3 req-87327a16-a441-49c7-a8b1-f99a807dbd76 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:02.359460 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-af7ff48c-0669-4b4a-9903-da57ad6dfbe3 req-87327a16-a441-49c7-a8b1-f99a807dbd76 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 943 time: 0.0799999 May 24 11:25:02.404363 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-af7ff48c-0669-4b4a-9903-da57ad6dfbe3 req-d602f476-3e27-429d-89eb-8a3f94edbdd3 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0392694 May 24 11:25:03.140370 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bfd3230a-18c5-49ec-9150-7334a93f5d3b req-a0102347-511c-4a01-8f97-a1c6b5a017c6 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:03.141860 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bfd3230a-18c5-49ec-9150-7334a93f5d3b req-a0102347-511c-4a01-8f97-a1c6b5a017c6 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 946 time: 0.1114495 May 24 11:25:03.190384 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bfd3230a-18c5-49ec-9150-7334a93f5d3b req-1251be52-97b4-4c54-8948-0c9444dee70d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0424626 May 24 11:25:03.611261 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40cb7451-1989-4f7d-ace4-c192080362d2 req-90d00dc3-a10b-4235-b159-12a11c9641d0 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:03.613863 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-40cb7451-1989-4f7d-ace4-c192080362d2 req-90d00dc3-a10b-4235-b159-12a11c9641d0 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 943 time: 0.0991373 May 24 11:25:03.670382 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-40cb7451-1989-4f7d-ace4-c192080362d2 req-d0e70544-16d3-4fb3-aa7c-200f1d940f8b tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0474880 May 24 11:25:04.467213 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07c04d89-67d9-4f24-b0c6-fbe0a076afca req-43568c0a-3373-4cea-8cf1-78a4cb0d4280 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:04.469823 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-07c04d89-67d9-4f24-b0c6-fbe0a076afca req-43568c0a-3373-4cea-8cf1-78a4cb0d4280 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 946 time: 0.1138115 May 24 11:25:04.525765 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-07c04d89-67d9-4f24-b0c6-fbe0a076afca req-30591358-747a-4150-adef-01ba54d20014 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0495765 May 24 11:25:04.857959 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6edc62d8-616a-4552-85bd-3ce03c1b434b req-9c445136-33d2-4fad-a754-e4643c2c897c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:04.859181 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6edc62d8-616a-4552-85bd-3ce03c1b434b req-9c445136-33d2-4fad-a754-e4643c2c897c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 943 time: 0.0811908 May 24 11:25:04.915545 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6edc62d8-616a-4552-85bd-3ce03c1b434b req-b5335440-505d-4689-bc25-71c997948f60 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0493360 May 24 11:25:06.123445 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d94c1839-4a6d-4770-aed8-ce21a8571ba0 req-5a538214-531c-4d59-b3e6-ebbca185473c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:06.124959 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d94c1839-4a6d-4770-aed8-ce21a8571ba0 req-5a538214-531c-4d59-b3e6-ebbca185473c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 943 time: 0.0835283 May 24 11:25:06.174056 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d94c1839-4a6d-4770-aed8-ce21a8571ba0 req-67398971-5288-40e0-be32-797e18a641d4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0430698 May 24 11:25:07.314256 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device f7c7da3d-a1d5-44d2-a684-ba1255d063df details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:25:07.396880 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9e5a7a89-33ea-4d80-a960-cdbd82d80fbf req-d727ca57-7f51-4daa-8f0b-586d5e4ca412 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:07.398414 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9e5a7a89-33ea-4d80-a960-cdbd82d80fbf req-d727ca57-7f51-4daa-8f0b-586d5e4ca412 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 943 time: 0.0928569 May 24 11:25:07.448134 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9e5a7a89-33ea-4d80-a960-cdbd82d80fbf req-e35ade12-a320-4c52-b6ac-505e657ee773 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0403216 May 24 11:25:07.460488 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port f7c7da3d-a1d5-44d2-a684-ba1255d063df, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=f7c7da3d-a1d5-44d2-a684-ba1255d063df,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:25:07.475401 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'port_id': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df', 'mac_address': 'fa:16:3e:35:63:c5', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 503, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.3'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:25:07.554977 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:07.573126 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id f7c7da3d-a1d5-44d2-a684-ba1255d063df {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:25:07.589329 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port f7c7da3d-a1d5-44d2-a684-ba1255d063df, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=f7c7da3d-a1d5-44d2-a684-ba1255d063df,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:25:07.594312 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:07.596969 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:07.708831 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=f7c7da3d-a1d5-44d2-a684-ba1255d063df,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:07.712011 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.115s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:07.810775 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['f7c7da3d-a1d5-44d2-a684-ba1255d063df'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:25:08.650088 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9f7f9295-3f89-441a-8575-ac54cac4d775 req-9e1ae43a-75e4-4509-a1c5-e1ae9dc5b488 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:08.651619 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9f7f9295-3f89-441a-8575-ac54cac4d775 req-9e1ae43a-75e4-4509-a1c5-e1ae9dc5b488 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 944 time: 0.0895133 May 24 11:25:08.698645 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9f7f9295-3f89-441a-8575-ac54cac4d775 req-00dfdc73-d1b8-4810-bdaa-c6005ecebc9b tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0409167 May 24 11:25:09.337712 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device f7c7da3d-a1d5-44d2-a684-ba1255d063df up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:25:09.430495 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port f7c7da3d-a1d5-44d2-a684-ba1255d063df completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:25:09.438710 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port f7c7da3d-a1d5-44d2-a684-ba1255d063df triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:25:09.439161 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:09.578320 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:09.609447 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:25:09.620003 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port f7c7da3d-a1d5-44d2-a684-ba1255d063df, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=f7c7da3d-a1d5-44d2-a684-ba1255d063df,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:25:09.626713 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:09.631304 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:09.704891 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df', 'code': 200} May 24 11:25:09.763058 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=f7c7da3d-a1d5-44d2-a684-ba1255d063df,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:09.769335 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.138s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:10.074113 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-53727c4a-e979-4162-8480-65cc47a0fc1d req-cd40dc21-730c-41cf-a06a-bd0f78d191f7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:10.075574 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-53727c4a-e979-4162-8480-65cc47a0fc1d req-cd40dc21-730c-41cf-a06a-bd0f78d191f7 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 1098 time: 0.1194487 May 24 11:25:10.085132 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ac5b9992-434e-4877-9fe7-6fbdf4c9e607 req-049e54f2-a3a8-4fa9-bc36-65a57be3f698 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:10.086651 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ac5b9992-434e-4877-9fe7-6fbdf4c9e607 req-049e54f2-a3a8-4fa9-bc36-65a57be3f698 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 945 time: 0.1814220 May 24 11:25:10.156145 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-53727c4a-e979-4162-8480-65cc47a0fc1d req-580c8aa3-be4d-47b3-80ee-46e13254625c service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0685194 May 24 11:25:10.161052 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ac5b9992-434e-4877-9fe7-6fbdf4c9e607 req-6f5e501e-76f6-4a67-bb99-ae75708016c1 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0609298 May 24 11:25:11.355185 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a8eb3e46-3e12-4716-a1f3-8982bb2ba577 req-f43e74b3-f9a9-4afc-8b41-c1a62dba2ac4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:11.356799 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a8eb3e46-3e12-4716-a1f3-8982bb2ba577 req-f43e74b3-f9a9-4afc-8b41-c1a62dba2ac4 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 945 time: 0.0828018 May 24 11:25:11.406078 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a8eb3e46-3e12-4716-a1f3-8982bb2ba577 req-b03f4d92-747f-44ac-bb97-0215895419ac tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0435126 May 24 11:25:11.604187 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a80c8a2f-5e17-440d-a051-00d50768b4aa req-474c7df9-0dd7-4ed3-954d-4451387b58b0 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:11.605670 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a80c8a2f-5e17-440d-a051-00d50768b4aa req-474c7df9-0dd7-4ed3-954d-4451387b58b0 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 945 time: 0.0916038 May 24 11:25:11.656152 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a80c8a2f-5e17-440d-a051-00d50768b4aa req-8d356833-0dd8-44a9-b497-839c35a936ee tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0443580 May 24 11:25:11.680931 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 32990) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:11.765923 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-f6cb2c88-e835-4f20-b5b1-ef2e2ff03d88 tempest-TestVolumeBootPattern-1615708037 tempest-TestVolumeBootPattern-1615708037-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:11.766532 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-f6cb2c88-e835-4f20-b5b1-ef2e2ff03d88 tempest-TestVolumeBootPattern-1615708037 tempest-TestVolumeBootPattern-1615708037-project-admin] Synchronizing usage tracker for tenant:672c5f4985084a8492a86667aceb0768 on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:25:11.783678 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-f6cb2c88-e835-4f20-b5b1-ef2e2ff03d88 tempest-TestVolumeBootPattern-1615708037 tempest-TestVolumeBootPattern-1615708037-project-admin] Unset dirty status for tenant:672c5f4985084a8492a86667aceb0768 on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:25:11.785312 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-f6cb2c88-e835-4f20-b5b1-ef2e2ff03d88 tempest-TestVolumeBootPattern-1615708037 tempest-TestVolumeBootPattern-1615708037-project-admin] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 1093 time: 0.1032639 May 24 11:25:11.788347 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 32994) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:11.794275 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Request body: {'floatingip': {'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'port_id': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address': '172.20.0.3'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:25:11.800335 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:25:11.804205 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Usage tracker for resource:floatingip and tenant:9a8d61f79bbc496ca30e390fd9b5839e is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:25:11.807178 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Quota usage for floatingip was recalculated. Used quota:2. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:25:11.813792 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attempting to reserve 1 items for resource floatingip. Total usage: 2; quota limit: 50; headroom:48 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:25:11.823647 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:12.007617 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:12.154201 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Allocated IP 172.27.0.4 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/f452b93f-3e52-4dd7-8f71-f1fea984923a) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:25:12.338321 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:12.368812 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:12.372567 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-93819d28-8f57-4f88-9874-757fe4c676b9 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:12.435610 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:25:12.438327 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:25:12.442934 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:25:12.476043 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'Port': ['ID=f452b93f-3e52-4dd7-8f71-f1fea984923a,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:12.482140 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.110s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:12.997984 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:13.077387 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:13.292712 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:13.297450 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:13.302507 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-81199ec1-6e9e-4f7c-afb5-c1967d59c458 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:13.415587 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event updated for resources: {'Port': ['ID=f452b93f-3e52-4dd7-8f71-f1fea984923a,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:13.418455 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:13.426551 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:13.603309 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Floating IP f469c6c5-0e2c-4d4c-8124-71782bcea9ea associated. External IP: 172.27.0.4, port: f7c7da3d-a1d5-44d2-a684-ba1255d063df. May 24 11:25:13.965272 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Router 81705dbf-841d-45b4-8b2d-afa9886a8e59 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:25:14.251564 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:25:14.261404 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6bcc0276-30b5-409c-8400-7a2fc2154f86 None None] Sync routers for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:25:14.271103 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6bcc0276-30b5-409c-8400-7a2fc2154f86 None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:25:14.271692 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-6bcc0276-30b5-409c-8400-7a2fc2154f86 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:24:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:14.278250 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:14.333181 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 994 time: 2.5435021 May 24 11:25:14.333821 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'tag': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:25:14.381338 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'tag': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df', 'status': 'completed', 'code': 200} May 24 11:25:14.477831 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c8970aca-0677-4d91-a0ab-b756775f9186 req-5d1b33a0-7885-48fc-9ce1-2f0d2c70dbb5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:14.479342 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c8970aca-0677-4d91-a0ab-b756775f9186 req-5d1b33a0-7885-48fc-9ce1-2f0d2c70dbb5 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 1098 time: 0.0731063 May 24 11:25:14.486973 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-6bcc0276-30b5-409c-8400-7a2fc2154f86 None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:25:14.612036 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c8970aca-0677-4d91-a0ab-b756775f9186 req-66657f36-178c-4c07-bd68-a9fd9ec70baf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:14.613314 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c8970aca-0677-4d91-a0ab-b756775f9186 req-66657f36-178c-4c07-bd68-a9fd9ec70baf service neutron] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 912 time: 0.1274018 May 24 11:25:14.722602 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c8970aca-0677-4d91-a0ab-b756775f9186 req-a0d23ed7-adaf-478a-a985-432bf65fcd1a service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=f7c7da3d-a1d5-44d2-a684-ba1255d063df HTTP/1.1" status: 200 len: 997 time: 0.1032672 May 24 11:25:14.794576 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c8970aca-0677-4d91-a0ab-b756775f9186 req-cdab5ae1-5a12-4ef6-92a9-7859ad3a9150 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:14.796021 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c8970aca-0677-4d91-a0ab-b756775f9186 req-cdab5ae1-5a12-4ef6-92a9-7859ad3a9150 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.0661116 May 24 11:25:14.864544 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-6bcc0276-30b5-409c-8400-7a2fc2154f86 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': 'bfd25b24-9190-4f5b-be56-6f7de757deb6', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.8', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'fixed_ip_address': '172.20.0.9', 'status': 'ACTIVE', 'standard_attr_id': 534, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:a6:bc:9e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '45596010-5877-4cfc-8808-752c422ae93a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:23:06Z', 'updated_at': '2021-05-24T11:23:09Z', 'revision_number': 1, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}, {'id': 'f469c6c5-0e2c-4d4c-8124-71782bcea9ea', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.4', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 606, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:35:63:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:25:12Z', 'updated_at': '2021-05-24T11:25:12Z', 'revision_number': 0, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:14.868762 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6bcc0276-30b5-409c-8400-7a2fc2154f86 None None] Checking router: 81705dbf-841d-45b4-8b2d-afa9886a8e59 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:25:14.874434 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c8970aca-0677-4d91-a0ab-b756775f9186 req-c238c987-8473-48c7-a51a-79a1f68b930a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:14.876023 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c8970aca-0677-4d91-a0ab-b756775f9186 req-c238c987-8473-48c7-a51a-79a1f68b930a service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0736115 May 24 11:25:14.880365 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6bcc0276-30b5-409c-8400-7a2fc2154f86 None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:25:14.880963 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-6bcc0276-30b5-409c-8400-7a2fc2154f86 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:24:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:15.021164 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c8970aca-0677-4d91-a0ab-b756775f9186 req-4902342a-d5d5-4b45-b3b2-ea5303372152 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1387753 May 24 11:25:15.070463 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-6bcc0276-30b5-409c-8400-7a2fc2154f86 None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:25:15.163924 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-c8970aca-0677-4d91-a0ab-b756775f9186 req-65866eb9-8914-46c8-813e-95a01221dae7 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1334383 May 24 11:25:15.443528 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-6bcc0276-30b5-409c-8400-7a2fc2154f86 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': 'bfd25b24-9190-4f5b-be56-6f7de757deb6', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.8', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'fixed_ip_address': '172.20.0.9', 'status': 'ACTIVE', 'standard_attr_id': 534, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:a6:bc:9e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '45596010-5877-4cfc-8808-752c422ae93a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:23:06Z', 'updated_at': '2021-05-24T11:23:09Z', 'revision_number': 1, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}, {'id': 'f469c6c5-0e2c-4d4c-8124-71782bcea9ea', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.4', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 606, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:35:63:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:25:12Z', 'updated_at': '2021-05-24T11:25:12Z', 'revision_number': 0, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:15.447868 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6bcc0276-30b5-409c-8400-7a2fc2154f86 None None] The sync data for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi is: [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': 'bfd25b24-9190-4f5b-be56-6f7de757deb6', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.8', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'fixed_ip_address': '172.20.0.9', 'status': 'ACTIVE', 'standard_attr_id': 534, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:a6:bc:9e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '45596010-5877-4cfc-8808-752c422ae93a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:23:06Z', 'updated_at': '2021-05-24T11:23:09Z', 'revision_number': 1, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}, {'id': 'f469c6c5-0e2c-4d4c-8124-71782bcea9ea', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.4', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 606, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:35:63:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:25:12Z', 'updated_at': '2021-05-24T11:25:12Z', 'revision_number': 0, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:25:15.641119 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ebaaad80-8ad2-4325-b052-e5a72021a41e None None] New status for floating IP f469c6c5-0e2c-4d4c-8124-71782bcea9ea: ACTIVE {{(pid=91821) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:25:21.967438 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b5a25f70-8d33-4162-aec0-65401448d948 req-b99fbb22-40bb-4ec2-a164-654d62fd42a2 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0742037 May 24 11:25:22.059965 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5a25f70-8d33-4162-aec0-65401448d948 req-1ca47a66-fb5d-4601-ac87-767bae7dccc6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:22.061456 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b5a25f70-8d33-4162-aec0-65401448d948 req-1ca47a66-fb5d-4601-ac87-767bae7dccc6 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 1099 time: 0.0723085 May 24 11:25:22.067675 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:22.211692 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5a25f70-8d33-4162-aec0-65401448d948 req-32d4dcff-63eb-4e42-a9bb-63594f991da9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:22.213314 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b5a25f70-8d33-4162-aec0-65401448d948 req-32d4dcff-63eb-4e42-a9bb-63594f991da9 service neutron] 10.222.0.93 "GET /v2.0/networks?id=7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 200 len: 917 time: 0.1314418 May 24 11:25:22.319301 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b5a25f70-8d33-4162-aec0-65401448d948 req-62ab5041-1d56-4299-b532-48cc598f4576 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=a92e9868-2cc3-4d24-859b-18840b6fce56 HTTP/1.1" status: 200 len: 1001 time: 0.0975249 May 24 11:25:22.388444 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5a25f70-8d33-4162-aec0-65401448d948 req-c4fe670b-c1e0-41ec-ae0a-84011075b56a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:22.389922 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b5a25f70-8d33-4162-aec0-65401448d948 req-c4fe670b-c1e0-41ec-ae0a-84011075b56a service neutron] 10.222.0.93 "GET /v2.0/subnets?id=12ff35e8-9af8-4339-95c9-c62c90d8c6fd HTTP/1.1" status: 200 len: 855 time: 0.0593050 May 24 11:25:22.480451 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5a25f70-8d33-4162-aec0-65401448d948 req-c9c72b5c-6aba-4e1d-a7d2-3d7926276d9c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:22.481970 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b5a25f70-8d33-4162-aec0-65401448d948 req-c9c72b5c-6aba-4e1d-a7d2-3d7926276d9c service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=7877e643-fe9e-4041-91fa-3a4f44279fd2&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0840552 May 24 11:25:22.724191 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b5a25f70-8d33-4162-aec0-65401448d948 req-a7e94c78-8842-488c-9bb9-57a36aab31e6 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2347946 May 24 11:25:22.868288 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b5a25f70-8d33-4162-aec0-65401448d948 req-04a2470a-0102-4de9-8868-f57b238e9392 service neutron] 10.222.0.93 "GET /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1372395 May 24 11:25:23.428115 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-4c0bd84d-d3d0-4f9d-ad3b-ff32dff6dc79 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:25.758417 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:25.877747 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f511305f-7768-419c-9181-c123e3bc0207 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:26.997278 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:28.946514 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:34.293908 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:25:37.810613 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a88c075a-5eca-4b51-a209-1293013be19e req-a2108af9-e9f3-4347-9d05-c653a4aa06de tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:37.811976 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a88c075a-5eca-4b51-a209-1293013be19e req-a2108af9-e9f3-4347-9d05-c653a4aa06de tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 946 time: 0.0802948 May 24 11:25:37.856450 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-a88c075a-5eca-4b51-a209-1293013be19e req-e8a8417e-a21d-40ae-a854-1ac4390b555a tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0382278 May 24 11:25:39.123049 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e135fda-f6d3-4e61-a9a4-1e413b9a2c85 req-ef83d2c8-b229-46a4-aaaf-80b679f87534 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:39.124577 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4e135fda-f6d3-4e61-a9a4-1e413b9a2c85 req-ef83d2c8-b229-46a4-aaaf-80b679f87534 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 946 time: 0.1424427 May 24 11:25:39.132851 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-f80253e8-de2a-43a8-9090-eb56b32dbdba tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:25:39.134265 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-f80253e8-de2a-43a8-9090-eb56b32dbdba tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.93 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 946 time: 0.1203973 May 24 11:25:39.258714 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4e135fda-f6d3-4e61-a9a4-1e413b9a2c85 req-fb93015b-7076-44dd-962a-19a0ccc90381 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups?id=07d7df74-cc03-44ab-b535-c3ef7f5a7611&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1234069 May 24 11:25:39.314338 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Deleting port a92e9868-2cc3-4d24-859b-18840b6fce56 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:25:39.314672 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:39.470466 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] For port a92e9868-2cc3-4d24-859b-18840b6fce56, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=a92e9868-2cc3-4d24-859b-18840b6fce56,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:25:39.470817 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:39.779530 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:39.779907 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:39.780227 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Floating IP 1f2903d8-b12d-4d40-bb9f-84aa6c3e7266 disassociated. External IP: 172.27.0.143, port: a92e9868-2cc3-4d24-859b-18840b6fce56. May 24 11:25:39.780570 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Calling delete_port for a92e9868-2cc3-4d24-859b-18840b6fce56 owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:25:39.914142 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:39.920739 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-19c23cb5-cbe7-4bba-9024-ec78a1e3fb6c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:39.967401 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event deleted for resources: {'Port': ['ID=a92e9868-2cc3-4d24-859b-18840b6fce56,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:39.971001 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:40.173601 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Router c2ffed60-7ad5-4215-9e01-680707fb98d6 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:25:40.405360 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:25:40.418307 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c9fcd039-8f0e-4c4a-9fda-e477cfe340b7 None None] Sync routers for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:25:40.440408 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-10bc56f6-be94-4c60-bda8-a753c872d188 req-ffa02aa5-e5f1-4136-80b8-0467dd6a56b6 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/ports?device_id=b0cd69d0-e09d-489b-8d01-7c2042af18ed HTTP/1.1" status: 200 len: 210 time: 0.0540371 May 24 11:25:40.444201 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c9fcd039-8f0e-4c4a-9fda-e477cfe340b7 None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:25:40.446020 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-c9fcd039-8f0e-4c4a-9fda-e477cfe340b7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:25:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:40.619666 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] No DVR routers for this DVR port a92e9868-2cc3-4d24-859b-18840b6fce56 on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:25:40.622298 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:40.625260 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'name': 'network-vif-deleted', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:25:40.645769 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-23d0ea81-6971-4ad2-a248-b41f2edc1159 req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.93 "DELETE /v2.0/ports/a92e9868-2cc3-4d24-859b-18840b6fce56 HTTP/1.1" status: 204 len: 173 time: 1.4974504 May 24 11:25:40.681692 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'b0cd69d0-e09d-489b-8d01-7c2042af18ed', 'name': 'network-vif-deleted', 'tag': 'a92e9868-2cc3-4d24-859b-18840b6fce56', 'status': 'completed', 'code': 200} May 24 11:25:40.727369 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-c9fcd039-8f0e-4c4a-9fda-e477cfe340b7 None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:25:41.099478 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-c9fcd039-8f0e-4c4a-9fda-e477cfe340b7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:26Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:41.104220 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c9fcd039-8f0e-4c4a-9fda-e477cfe340b7 None None] Checking router: c2ffed60-7ad5-4215-9e01-680707fb98d6 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:25:41.113579 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c9fcd039-8f0e-4c4a-9fda-e477cfe340b7 None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:25:41.114209 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-c9fcd039-8f0e-4c4a-9fda-e477cfe340b7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:25:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:41.319738 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-c9fcd039-8f0e-4c4a-9fda-e477cfe340b7 None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:25:41.647426 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-c9fcd039-8f0e-4c4a-9fda-e477cfe340b7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:26Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:41.651482 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c9fcd039-8f0e-4c4a-9fda-e477cfe340b7 None None] The sync data for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi is: [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:20:20Z', 'revision_number': 4, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '9c46ec57-71f9-47d3-8520-7247c2991687', 'name': '', 'network_id': '7877e643-fe9e-4041-91fa-3a4f44279fd2', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'mac_address': 'fa:16:3e:2b:8e:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_interface', 'standard_attr_id': 450, 'fixed_ips': [{'subnet_id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:17Z', 'updated_at': '2021-05-24T11:20:26Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'subnets': [{'id': '12ff35e8-9af8-4339-95c9-c62c90d8c6fd', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:25:41.881824 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-beed94ef-1d90-4ef9-9dd8-a55bb255f8d6 None None] New status for floating IP 1f2903d8-b12d-4d40-bb9f-84aa6c3e7266: DOWN {{(pid=91821) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:25:42.608507 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33036) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:42.678074 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5d28e9fe-eca9-4134-ae0f-5b6eb56152d3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:42.709777 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5d28e9fe-eca9-4134-ae0f-5b6eb56152d3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:42.770451 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5d28e9fe-eca9-4134-ae0f-5b6eb56152d3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:42.774590 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-dadb47e1-d55f-4d02-a9a3-3a7b7c8b4990 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:42.791176 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5d28e9fe-eca9-4134-ae0f-5b6eb56152d3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=07d7df74-cc03-44ab-b535-c3ef7f5a7611,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:42.794176 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5d28e9fe-eca9-4134-ae0f-5b6eb56152d3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:42.807475 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-5d28e9fe-eca9-4134-ae0f-5b6eb56152d3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:42.809364 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-5d28e9fe-eca9-4134-ae0f-5b6eb56152d3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "DELETE /v2.0/security-groups/07d7df74-cc03-44ab-b535-c3ef7f5a7611 HTTP/1.1" status: 204 len: 168 time: 0.1990092 May 24 11:25:42.812024 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33040) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:42.831081 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.api.v2.resource [None req-bc74f733-0dc3-4947-a7af-a94a8ac5fb43 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] show failed (client error): The resource could not be found. May 24 11:25:42.832075 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-bc74f733-0dc3-4947-a7af-a94a8ac5fb43 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "GET /v2.0/security-groups/07d7df74-cc03-44ab-b535-c3ef7f5a7611 HTTP/1.1" status: 404 len: 347 time: 0.0188153 May 24 11:25:42.835037 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33044) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:43.059194 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-53403415-2785-42ad-a6f2-0a2fa896874d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Deleting port 8ab34aa2-04db-4ff9-88aa-b55bd6070a66 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:25:43.059429 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-53403415-2785-42ad-a6f2-0a2fa896874d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:43.177530 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-53403415-2785-42ad-a6f2-0a2fa896874d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:43.177854 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-53403415-2785-42ad-a6f2-0a2fa896874d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:43.299461 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-53403415-2785-42ad-a6f2-0a2fa896874d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Calling delete_port for 8ab34aa2-04db-4ff9-88aa-b55bd6070a66 owned by network:floatingip {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:25:43.416401 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-53403415-2785-42ad-a6f2-0a2fa896874d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:43.420635 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-74b83e04-d9a7-4ccf-bcf6-8df50a34204c tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:43.472603 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-53403415-2785-42ad-a6f2-0a2fa896874d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Pushing event deleted for resources: {'Port': ['ID=8ab34aa2-04db-4ff9-88aa-b55bd6070a66,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:43.475967 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-53403415-2785-42ad-a6f2-0a2fa896874d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:43.506528 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device a92e9868-2cc3-4d24-859b-18840b6fce56 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:25:43.542132 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: a92e9868-2cc3-4d24-859b-18840b6fce56 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:25:43.542485 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device a92e9868-2cc3-4d24-859b-18840b6fce56 not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:25:43.573430 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-53403415-2785-42ad-a6f2-0a2fa896874d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks [] for floatingip, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:43.587714 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-53403415-2785-42ad-a6f2-0a2fa896874d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:43.588871 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-53403415-2785-42ad-a6f2-0a2fa896874d tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] 10.222.0.18 "DELETE /v2.0/floatingips/1f2903d8-b12d-4d40-bb9f-84aa6c3e7266 HTTP/1.1" status: 204 len: 168 time: 0.7529635 May 24 11:25:46.395242 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33070) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:47.232613 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-687465'] for router_interface, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:47.645180 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] Deleting port 9c46ec57-71f9-47d3-8520-7247c2991687 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:25:47.645884 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:47.767942 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] For port 9c46ec57-71f9-47d3-8520-7247c2991687, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=9c46ec57-71f9-47d3-8520-7247c2991687,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:25:47.768290 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:47.891295 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] Calling delete_port for 9c46ec57-71f9-47d3-8520-7247c2991687 owned by network:router_interface {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:25:48.051160 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:48.054983 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-fbfd6ab9-b584-4e40-ab18-517649a2a5af tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:48.097250 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] Pushing event deleted for resources: {'Port': ['ID=9c46ec57-71f9-47d3-8520-7247c2991687,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:48.100313 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:48.391769 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-537767', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-685561'] for router_interface, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:48.867296 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] Router c2ffed60-7ad5-4215-9e01-680707fb98d6 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:25:49.045286 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:25:49.054545 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ee788ca2-84b5-4b83-8819-353ccb247a9c None None] Sync routers for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:25:49.058385 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] 10.222.0.18 "PUT /v2.0/routers/c2ffed60-7ad5-4215-9e01-680707fb98d6/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.6620743 May 24 11:25:49.060853 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33076) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:49.064636 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ee788ca2-84b5-4b83-8819-353ccb247a9c None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:25:49.065303 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-ee788ca2-84b5-4b83-8819-353ccb247a9c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:25:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:49.235668 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Notify callbacks [] for router, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:49.242004 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-ee788ca2-84b5-4b83-8819-353ccb247a9c None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:25:49.411537 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:49.482651 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Deleting port 839b9282-2a46-4561-9d08-f49bad628dc7 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:25:49.483097 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:49.531354 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-ee788ca2-84b5-4b83-8819-353ccb247a9c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:25:47Z', 'revision_number': 5, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port': {'id': '839b9282-2a46-4561-9d08-f49bad628dc7', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7d:7d:74', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'device_owner': 'network:router_gateway', 'standard_attr_id': 438, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:20:11Z', 'updated_at': '2021-05-24T11:20:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:49.535066 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ee788ca2-84b5-4b83-8819-353ccb247a9c None None] Checking router: c2ffed60-7ad5-4215-9e01-680707fb98d6 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:25:49.543530 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ee788ca2-84b5-4b83-8819-353ccb247a9c None None] Routers {'c2ffed60-7ad5-4215-9e01-680707fb98d6'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:25:49.544133 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-ee788ca2-84b5-4b83-8819-353ccb247a9c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:25:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['c2ffed60-7ad5-4215-9e01-680707fb98d6'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:49.604681 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] For port 839b9282-2a46-4561-9d08-f49bad628dc7, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=839b9282-2a46-4561-9d08-f49bad628dc7,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:25:49.605036 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:49.690409 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-ee788ca2-84b5-4b83-8819-353ccb247a9c None None] Set router c2ffed60-7ad5-4215-9e01-680707fb98d6 gateway port host: None {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:25:49.729539 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Calling delete_port for 839b9282-2a46-4561-9d08-f49bad628dc7 owned by network:router_gateway {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:25:49.829522 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-ee788ca2-84b5-4b83-8819-353ccb247a9c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:25:49Z', 'revision_number': 6, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port_host': None}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:49.833483 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ee788ca2-84b5-4b83-8819-353ccb247a9c None None] The sync data for ids ['c2ffed60-7ad5-4215-9e01-680707fb98d6'] in dv-cinder-784092-9-iscsi is: [{'id': 'c2ffed60-7ad5-4215-9e01-680707fb98d6', 'name': 'tempest-AttachVolumeShelveTestJSON-984022173-router', 'tenant_id': '05ec02ff35d2402e8d7bf064752e67ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:20:10Z', 'updated_at': '2021-05-24T11:25:49Z', 'revision_number': 6, 'project_id': '05ec02ff35d2402e8d7bf064752e67ea', 'gw_port_host': None}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:25:49.878460 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:49.883080 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-472f4c9d-44d2-4808-a57f-9c1259ec5bf5 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:49.962307 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Pushing event deleted for resources: {'Port': ['ID=839b9282-2a46-4561-9d08-f49bad628dc7,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:49.964783 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:50.021830 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-686964'] for router_gateway, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:50.297348 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-684537', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4171'] for router, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:50.299517 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:50.323004 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-684229'] for router, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:50.323611 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Fanout notify agent at l3_agent the message router_deleted on router c2ffed60-7ad5-4215-9e01-680707fb98d6 {{(pid=91819) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 11:25:50.336653 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] Notify callbacks [] for router, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:50.337808 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-03cbaaf4-422d-4918-b1dd-0cd8b95df270 admin admin] 10.222.0.18 "DELETE /v2.0/routers/c2ffed60-7ad5-4215-9e01-680707fb98d6 HTTP/1.1" status: 204 len: 168 time: 1.2760336 May 24 11:25:50.340332 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33080) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:50.402204 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_v2 [None req-fc980baf-3103-486f-ac74-56d7e2a542ac admin admin] Deleting subnet 12ff35e8-9af8-4339-95c9-c62c90d8c6fd {{(pid=91819) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:25:50.406002 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-fc980baf-3103-486f-ac74-56d7e2a542ac admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:50.454261 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-fc980baf-3103-486f-ac74-56d7e2a542ac admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534853172'] for subnet, precommit_delete_associations {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:50.534684 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-fc980baf-3103-486f-ac74-56d7e2a542ac admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:50.643099 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 9c46ec57-71f9-47d3-8520-7247c2991687 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:25:50.676897 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 9c46ec57-71f9-47d3-8520-7247c2991687 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:25:50.677289 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 9c46ec57-71f9-47d3-8520-7247c2991687 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:25:50.677727 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 839b9282-2a46-4561-9d08-f49bad628dc7 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:25:50.714422 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 839b9282-2a46-4561-9d08-f49bad628dc7 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:25:50.714783 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 839b9282-2a46-4561-9d08-f49bad628dc7 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:25:50.786614 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-fc980baf-3103-486f-ac74-56d7e2a542ac admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534944900', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535570805'] for subnet, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:50.790635 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-18f47b04-1672-4460-b676-b97133fb25ee admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:50.832554 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fc980baf-3103-486f-ac74-56d7e2a542ac admin admin] Pushing event deleted for resources: {'Subnet': ['ID=12ff35e8-9af8-4339-95c9-c62c90d8c6fd,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:50.835460 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-fc980baf-3103-486f-ac74-56d7e2a542ac admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:50.836435 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-fc980baf-3103-486f-ac74-56d7e2a542ac admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:50.841396 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-6f834d58-ebb8-415a-99ff-3518675aba12 None None] Network 7877e643-fe9e-4041-91fa-3a4f44279fd2 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:25:50.848287 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-fc980baf-3103-486f-ac74-56d7e2a542ac admin admin] 10.222.0.18 "DELETE /v2.0/subnets/12ff35e8-9af8-4339-95c9-c62c90d8c6fd HTTP/1.1" status: 204 len: 168 time: 0.5047750 May 24 11:25:50.848491 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33084) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:50.960147 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.037941 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Deleting port 9d9e4487-8f1c-4ba1-88e1-16045be7df72 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:25:51.038182 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.106195 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d37907ff-58ea-4955-971d-aa5e08408e1d None None] DHCP port deletion for 7877e643-fe9e-4041-91fa-3a4f44279fd2 request from dv-cinder-784092-9-iscsi {{(pid=91821) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 11:25:51.140873 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] For port 9d9e4487-8f1c-4ba1-88e1-16045be7df72, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=9d9e4487-8f1c-4ba1-88e1-16045be7df72,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:25:51.141258 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.174336 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-d37907ff-58ea-4955-971d-aa5e08408e1d None None] Deleting port 9d9e4487-8f1c-4ba1-88e1-16045be7df72 {{(pid=91821) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:25:51.174635 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d37907ff-58ea-4955-971d-aa5e08408e1d None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.263682 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Calling delete_port for 9d9e4487-8f1c-4ba1-88e1-16045be7df72 owned by network:dhcp {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:25:51.271647 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-d37907ff-58ea-4955-971d-aa5e08408e1d None None] For port 9d9e4487-8f1c-4ba1-88e1-16045be7df72, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=9d9e4487-8f1c-4ba1-88e1-16045be7df72,segment=NetworkSegment(31ae7943-96d7-4018-b76d-057ae76567e2),segment_id=31ae7943-96d7-4018-b76d-057ae76567e2)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:25:51.271933 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d37907ff-58ea-4955-971d-aa5e08408e1d None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.325230 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.329353 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-98283cdc-6d7e-4254-ad53-38167f6dbbd8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:51.379608 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Pushing event deleted for resources: {'Port': ['ID=9d9e4487-8f1c-4ba1-88e1-16045be7df72,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:51.382773 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:51.383559 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] No DVR routers for this DVR port 9d9e4487-8f1c-4ba1-88e1-16045be7df72 on host dv-cinder-784092-9-iscsi {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:25:51.438461 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-d37907ff-58ea-4955-971d-aa5e08408e1d None None] Calling delete_port for 9d9e4487-8f1c-4ba1-88e1-16045be7df72 owned by network:dhcp {{(pid=91821) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:25:51.483061 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d37907ff-58ea-4955-971d-aa5e08408e1d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093'] for port, after_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.488687 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:51.504076 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.517603 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['7877e643-fe9e-4041-91fa-3a4f44279fd2']}} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:51.529926 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '31ae7943-96d7-4018-b76d-057ae76567e2') {'for_net_delete': True} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:51.530300 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '31ae7943-96d7-4018-b76d-057ae76567e2') {} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:25:51.533741 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d37907ff-58ea-4955-971d-aa5e08408e1d None None] No DVR routers for this DVR port 9d9e4487-8f1c-4ba1-88e1-16045be7df72 on host dv-cinder-784092-9-iscsi {{(pid=91821) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:25:51.537702 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d37907ff-58ea-4955-971d-aa5e08408e1d None None] Pushing event deleted for resources: {'Port': ['ID=9d9e4487-8f1c-4ba1-88e1-16045be7df72,revision_number=None']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:51.539833 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-d37907ff-58ea-4955-971d-aa5e08408e1d None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:51.546208 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534853183'] for segment, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.565863 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.572011 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Releasing vlan 500 on physical network physnet1 to pool {{(pid=91819) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:25:51.572416 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535155490'] for segment, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: Traceback (most recent call last): May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: timer() May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: cb(*args, **kw) May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: return func(*args, **kwargs) May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: self._notify() May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: self.callback(batched_events) May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: event.method(event) May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: return f(self, *a, **k) May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._get(url).json() May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: return self.request(url, 'GET', **kwargs) May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:25:51.585615 dv-cinder-784092-9-iscsi neutron-server[91819]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:25:51.775111 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.778844 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-429b943c-5492-4157-b93f-e73881b3c3fd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:51.793068 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:51.804382 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Pushing event deleted for resources: {'Network': ['ID=7877e643-fe9e-4041-91fa-3a4f44279fd2,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:51.807633 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:51.819305 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] 10.222.0.18 "DELETE /v2.0/networks/7877e643-fe9e-4041-91fa-3a4f44279fd2 HTTP/1.1" status: 204 len: 168 time: 0.9697554 May 24 11:25:51.940709 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33090) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:51.989988 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-fae79832-6086-4c68-a866-61079543e5a5 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=05ec02ff35d2402e8d7bf064752e67ea&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0480583 May 24 11:25:51.992357 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33094) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:52.049277 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-612b6946-c85a-43af-9451-69d1c5baadc6 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:52.075942 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-612b6946-c85a-43af-9451-69d1c5baadc6 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:52.078413 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:52.131906 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-612b6946-c85a-43af-9451-69d1c5baadc6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:52.136435 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a7953674-524d-4bf1-8acb-b32390f11b0e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:52.149859 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-612b6946-c85a-43af-9451-69d1c5baadc6 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2361ea90-9180-4574-b91e-b236f5fc2de0,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:52.153309 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-612b6946-c85a-43af-9451-69d1c5baadc6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:52.156547 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-612b6946-c85a-43af-9451-69d1c5baadc6 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:52.157655 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-612b6946-c85a-43af-9451-69d1c5baadc6 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/2361ea90-9180-4574-b91e-b236f5fc2de0 HTTP/1.1" status: 204 len: 168 time: 0.1644018 May 24 11:25:52.653566 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 9d9e4487-8f1c-4ba1-88e1-16045be7df72 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:25:52.689915 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 9d9e4487-8f1c-4ba1-88e1-16045be7df72 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:25:52.690275 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 9d9e4487-8f1c-4ba1-88e1-16045be7df72 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:25:53.430386 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-2e730216-6470-477d-964f-9580f5199fe7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:53.540430 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33148) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:53.668575 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:53.683034 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:25:53.687614 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Usage tracker for resource:security_group_rule and tenant:514c0ef2dcf34cbdad2d2a3209f94c6f is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:25:53.690611 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:25:53.696327 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:25:53.775627 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:53.779286 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:53.787681 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-63202e0d-1f0b-487d-8ee2-65458f91c165 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:53.826800 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=514c0ef2dcf34cbdad2d2a3209f94c6f&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2849848 May 24 11:25:53.828439 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=759e213f-63d5-498a-98fe-38e3f2f84dda,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:53.829593 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33154) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:53.836294 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:53.836666 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:53.837027 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:53.837413 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:53.837783 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:53.838114 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:53.838461 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:53.838818 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:53.839208 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:53.839556 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:53.839999 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:53.840346 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:53.845237 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:53.889370 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a99bb5c0-93ff-4d68-bf54-46022fa46a62 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:53.914048 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a99bb5c0-93ff-4d68-bf54-46022fa46a62 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:53.967511 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a99bb5c0-93ff-4d68-bf54-46022fa46a62 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:53.971517 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-290e192c-6cd3-4e89-88be-6153876a8998 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:53.985587 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a99bb5c0-93ff-4d68-bf54-46022fa46a62 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=759e213f-63d5-498a-98fe-38e3f2f84dda,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:53.987854 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a99bb5c0-93ff-4d68-bf54-46022fa46a62 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:53.991529 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-a99bb5c0-93ff-4d68-bf54-46022fa46a62 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:53.992728 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-a99bb5c0-93ff-4d68-bf54-46022fa46a62 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/759e213f-63d5-498a-98fe-38e3f2f84dda HTTP/1.1" status: 204 len: 168 time: 0.1618617 May 24 11:25:55.766520 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:55.878715 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-05a8b284-597b-467f-8ed6-06349beb7fda None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:56.997901 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:58.712162 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33240) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:58.828955 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6489e443-54ba-44ea-8cc1-0fd067303f84 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:58.846986 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-6489e443-54ba-44ea-8cc1-0fd067303f84 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:25:58.850722 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-6489e443-54ba-44ea-8cc1-0fd067303f84 admin admin] Usage tracker for resource:security_group_rule and tenant:547f91d5f2d044b8afcbfc4ae40d2660 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:25:58.853457 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-6489e443-54ba-44ea-8cc1-0fd067303f84 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:25:58.859784 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-6489e443-54ba-44ea-8cc1-0fd067303f84 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:25:58.939606 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:58.941865 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6489e443-54ba-44ea-8cc1-0fd067303f84 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:58.945458 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6489e443-54ba-44ea-8cc1-0fd067303f84 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:58.953279 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-86007b7a-e0db-4075-8d08-7c4b144047f5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:58.994438 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-6489e443-54ba-44ea-8cc1-0fd067303f84 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=547f91d5f2d044b8afcbfc4ae40d2660&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2812099 May 24 11:25:58.995361 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6489e443-54ba-44ea-8cc1-0fd067303f84 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=66362353-9503-4a5d-b6dc-3a766ef19993,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:58.997197 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33246) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:58.999033 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6489e443-54ba-44ea-8cc1-0fd067303f84 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:59.056187 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:59.084687 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:59.142597 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:59.146291 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5d28e9fe-eca9-4134-ae0f-5b6eb56152d3 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:59.159759 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=66362353-9503-4a5d-b6dc-3a766ef19993,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:59.169010 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:59.169408 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:59.169769 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:59.170123 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:59.171034 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:59.171405 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:59.171746 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:59.172098 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:59.172442 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:59.172787 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:59.173158 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:59.173503 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:25:59.176475 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:59.177445 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:59.179064 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-7533aa64-dea2-443d-b41a-5e66419639a6 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/66362353-9503-4a5d-b6dc-3a766ef19993 HTTP/1.1" status: 204 len: 168 time: 0.1801333 May 24 11:25:59.456052 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33254) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:59.479508 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6b8c93aa-053e-42ea-895a-0cbf7d6d46dd admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:59.494106 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-6b8c93aa-053e-42ea-895a-0cbf7d6d46dd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:25:59.498050 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-6b8c93aa-053e-42ea-895a-0cbf7d6d46dd admin admin] Usage tracker for resource:security_group_rule and tenant:f48bff2caf7d47dfba3550a574af2576 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:25:59.501825 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-6b8c93aa-053e-42ea-895a-0cbf7d6d46dd admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:25:59.507465 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-6b8c93aa-053e-42ea-895a-0cbf7d6d46dd admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:25:59.588977 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6b8c93aa-053e-42ea-895a-0cbf7d6d46dd admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:59.592768 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-6b8c93aa-053e-42ea-895a-0cbf7d6d46dd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:59.599453 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6489e443-54ba-44ea-8cc1-0fd067303f84 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:59.644273 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-6b8c93aa-053e-42ea-895a-0cbf7d6d46dd admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=f48bff2caf7d47dfba3550a574af2576&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1871362 May 24 11:25:59.645190 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6b8c93aa-053e-42ea-895a-0cbf7d6d46dd admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=eeb6beb4-4708-431c-b0ff-743aeec5d08d,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:59.647011 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33258) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:25:59.648136 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6b8c93aa-053e-42ea-895a-0cbf7d6d46dd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:59.704646 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-fa5c0701-a56b-478f-ad80-0e3e321921b0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:59.729938 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-fa5c0701-a56b-478f-ad80-0e3e321921b0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:59.783607 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-fa5c0701-a56b-478f-ad80-0e3e321921b0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:59.788274 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-0052ec49-c5c2-4727-b543-4f230606fb38 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:25:59.800491 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fa5c0701-a56b-478f-ad80-0e3e321921b0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=eeb6beb4-4708-431c-b0ff-743aeec5d08d,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:25:59.802849 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-fa5c0701-a56b-478f-ad80-0e3e321921b0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:25:59.806295 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-fa5c0701-a56b-478f-ad80-0e3e321921b0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:25:59.808079 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-fa5c0701-a56b-478f-ad80-0e3e321921b0 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/eeb6beb4-4708-431c-b0ff-743aeec5d08d HTTP/1.1" status: 204 len: 168 time: 0.1600890 May 24 11:26:01.610886 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33318) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:26:01.728940 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1711f0ae-7aa5-4c93-b08f-fdee8038f36d admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:01.747216 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-1711f0ae-7aa5-4c93-b08f-fdee8038f36d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:26:01.751578 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-1711f0ae-7aa5-4c93-b08f-fdee8038f36d admin admin] Usage tracker for resource:security_group_rule and tenant:e9a6fc2985284c378417d9079b53fa54 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:26:01.754403 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-1711f0ae-7aa5-4c93-b08f-fdee8038f36d admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:26:01.760475 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-1711f0ae-7aa5-4c93-b08f-fdee8038f36d admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:26:01.846805 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1711f0ae-7aa5-4c93-b08f-fdee8038f36d admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:01.852054 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1711f0ae-7aa5-4c93-b08f-fdee8038f36d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:01.859956 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-0575b8ff-47e2-456d-bf22-1817ba86ce09 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:26:01.901413 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-1711f0ae-7aa5-4c93-b08f-fdee8038f36d admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=e9a6fc2985284c378417d9079b53fa54&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2894244 May 24 11:26:01.902269 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1711f0ae-7aa5-4c93-b08f-fdee8038f36d admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f708bf45-096a-435f-8aa2-509071206dc3,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:26:01.904187 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33324) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:26:01.905683 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-1711f0ae-7aa5-4c93-b08f-fdee8038f36d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:26:01.967162 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-26105ded-e2d2-4bc9-9817-62e1663f912c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:01.994240 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-26105ded-e2d2-4bc9-9817-62e1663f912c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:02.053468 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-26105ded-e2d2-4bc9-9817-62e1663f912c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:02.057141 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-598b902e-ce41-4cb5-930b-99a38c6fc3fb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:26:02.071910 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-26105ded-e2d2-4bc9-9817-62e1663f912c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f708bf45-096a-435f-8aa2-509071206dc3,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:26:02.075823 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-26105ded-e2d2-4bc9-9817-62e1663f912c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:26:02.089115 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-26105ded-e2d2-4bc9-9817-62e1663f912c admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:02.090684 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-26105ded-e2d2-4bc9-9817-62e1663f912c admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/f708bf45-096a-435f-8aa2-509071206dc3 HTTP/1.1" status: 204 len: 168 time: 0.1846406 May 24 11:26:02.384652 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33332) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:26:02.403122 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-745823f6-e7cb-4386-a1e5-dfe1eebacaff admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:02.418634 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-745823f6-e7cb-4386-a1e5-dfe1eebacaff admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:26:02.423295 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-745823f6-e7cb-4386-a1e5-dfe1eebacaff admin admin] Usage tracker for resource:security_group_rule and tenant:3dbf8c635ef34d2cabfc5bd994818267 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:26:02.426234 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-745823f6-e7cb-4386-a1e5-dfe1eebacaff admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:26:02.432028 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-745823f6-e7cb-4386-a1e5-dfe1eebacaff admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:26:02.524198 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-745823f6-e7cb-4386-a1e5-dfe1eebacaff admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:02.528252 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-745823f6-e7cb-4386-a1e5-dfe1eebacaff admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:02.537136 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-41f656a5-9532-4de0-9691-bfe74c7625fb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:26:02.582940 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-745823f6-e7cb-4386-a1e5-dfe1eebacaff admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=38f98cbf-483b-47ed-adb4-e02e7de6623c,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:26:02.586174 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-745823f6-e7cb-4386-a1e5-dfe1eebacaff admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:26:02.588724 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-745823f6-e7cb-4386-a1e5-dfe1eebacaff admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=3dbf8c635ef34d2cabfc5bd994818267&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2029898 May 24 11:26:02.591419 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33336) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:26:02.652863 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ec483f4a-d564-4419-a290-eb1922d827ba admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:02.682734 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ec483f4a-d564-4419-a290-eb1922d827ba admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:02.738643 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ec483f4a-d564-4419-a290-eb1922d827ba admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:02.742372 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-916b852d-34f1-4b4e-bb52-229a63e72679 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:26:02.755173 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ec483f4a-d564-4419-a290-eb1922d827ba admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=38f98cbf-483b-47ed-adb4-e02e7de6623c,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:26:02.757797 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ec483f4a-d564-4419-a290-eb1922d827ba admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:26:02.765890 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ec483f4a-d564-4419-a290-eb1922d827ba admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:02.768110 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-ec483f4a-d564-4419-a290-eb1922d827ba admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/38f98cbf-483b-47ed-adb4-e02e7de6623c HTTP/1.1" status: 204 len: 168 time: 0.1740427 May 24 11:26:11.296253 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:26:22.096529 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:23.432467 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-ac3c0a5a-c6b0-4e5c-bb9b-92904f940255 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:25.747451 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-8db94881-0c81-47f4-b4eb-7b7840669551 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0880244 May 24 11:26:25.767531 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:25.881280 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-ecf56b11-d94d-44f7-b78b-1c167681ad8a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:25.900815 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-63a385d6-752d-4800-90c7-16e36e67e53e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:25.902574 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-63a385d6-752d-4800-90c7-16e36e67e53e service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 1098 time: 0.0846262 May 24 11:26:26.055478 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-93808a44-b824-45d2-9cc0-fca4ba1d2ad9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:26.056950 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-93808a44-b824-45d2-9cc0-fca4ba1d2ad9 service neutron] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 912 time: 0.1353366 May 24 11:26:26.165521 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-5e4381e4-21da-4f61-963e-d7f2db7e5942 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=8ca4165e-4c84-4940-9148-3f5a77eaf348 HTTP/1.1" status: 200 len: 999 time: 0.1025114 May 24 11:26:26.235063 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-7c0e4930-c23c-4bdd-a906-7b27ddb0428e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:26.236632 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-7c0e4930-c23c-4bdd-a906-7b27ddb0428e service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.0653167 May 24 11:26:26.322546 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-1bc97b1f-4739-424a-a121-8525fb26f75a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:26.324382 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-1bc97b1f-4739-424a-a121-8525fb26f75a service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0812252 May 24 11:26:26.335459 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-70d52b5d-d03f-402c-84d0-386c99e316f5 service neutron] http://10.222.0.18:9696/v2.0/extensions returned with HTTP 200 {{(pid=91819) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 11:26:26.336587 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-70d52b5d-d03f-402c-84d0-386c99e316f5 service neutron] 10.222.0.93 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0061557 May 24 11:26:26.470169 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-d4db0d56-5e0b-4ed5-8e05-f136ff318bca service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1271939 May 24 11:26:26.602824 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f97bbbbf-03cc-46c4-bdae-0de1fe2c3769 req-d70b9b52-02fa-4bc9-b3c3-e06f9910243f service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1254814 May 24 11:26:27.000630 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:28.944486 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:37.909203 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33530) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:26:38.031454 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-de839e3b-2579-4f78-969c-065d8e5e782d admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:38.045813 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-de839e3b-2579-4f78-969c-065d8e5e782d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:26:38.049749 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-de839e3b-2579-4f78-969c-065d8e5e782d admin admin] Usage tracker for resource:security_group_rule and tenant:65df17ad1ce74107b7233f860815dcf9 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:26:38.052492 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-de839e3b-2579-4f78-969c-065d8e5e782d admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:26:38.058206 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-de839e3b-2579-4f78-969c-065d8e5e782d admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:26:38.143166 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-de839e3b-2579-4f78-969c-065d8e5e782d admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:38.147189 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-de839e3b-2579-4f78-969c-065d8e5e782d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:38.154905 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-1711f0ae-7aa5-4c93-b08f-fdee8038f36d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:26:38.196771 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-de839e3b-2579-4f78-969c-065d8e5e782d admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=65df17ad1ce74107b7233f860815dcf9&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2864554 May 24 11:26:38.197570 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-de839e3b-2579-4f78-969c-065d8e5e782d admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=96c9b6c8-b402-47ec-adcc-25e008015bc4,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:26:38.200680 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33536) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:26:38.201479 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-de839e3b-2579-4f78-969c-065d8e5e782d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:26:38.262629 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ed01bcf9-ebb7-4c2a-a777-1e636b7a8f58 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:38.288840 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ed01bcf9-ebb7-4c2a-a777-1e636b7a8f58 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:38.347798 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ed01bcf9-ebb7-4c2a-a777-1e636b7a8f58 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:38.351756 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ee0d752c-3c2a-4f39-807e-c09c3c6e85d7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:26:38.364016 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ed01bcf9-ebb7-4c2a-a777-1e636b7a8f58 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=96c9b6c8-b402-47ec-adcc-25e008015bc4,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:26:38.366434 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ed01bcf9-ebb7-4c2a-a777-1e636b7a8f58 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:26:38.373059 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ed01bcf9-ebb7-4c2a-a777-1e636b7a8f58 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:38.374216 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-ed01bcf9-ebb7-4c2a-a777-1e636b7a8f58 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/96c9b6c8-b402-47ec-adcc-25e008015bc4 HTTP/1.1" status: 204 len: 168 time: 0.1718564 May 24 11:26:41.983934 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33584) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:26:42.354772 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Deleting port f452b93f-3e52-4dd7-8f71-f1fea984923a {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:26:42.355028 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:42.477853 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:42.478184 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:42.613432 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Calling delete_port for f452b93f-3e52-4dd7-8f71-f1fea984923a owned by network:floatingip {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:26:42.773420 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:42.774079 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-d9d2cc43-75b2-4409-a80d-691221cad7cd tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:26:42.857296 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event deleted for resources: {'Port': ['ID=f452b93f-3e52-4dd7-8f71-f1fea984923a,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:26:42.904038 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:26:43.009935 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:43.010637 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.l3_db [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Floating IP f469c6c5-0e2c-4d4c-8124-71782bcea9ea disassociated (deleted). External IP: 172.27.0.4, port: f7c7da3d-a1d5-44d2-a684-ba1255d063df. May 24 11:26:43.404390 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Router 81705dbf-841d-45b4-8b2d-afa9886a8e59 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:26:43.594712 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:26:43.604657 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-46551a1c-5d6d-46d7-8e28-f55cea9ad787 None None] Sync routers for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:26:43.616936 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-46551a1c-5d6d-46d7-8e28-f55cea9ad787 None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:26:43.617704 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-46551a1c-5d6d-46d7-8e28-f55cea9ad787 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:26:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:26:43.618799 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:43.676253 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "DELETE /v2.0/floatingips/f469c6c5-0e2c-4d4c-8124-71782bcea9ea HTTP/1.1" status: 204 len: 168 time: 1.6911881 May 24 11:26:43.676945 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'tag': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:26:43.836100 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-46551a1c-5d6d-46d7-8e28-f55cea9ad787 None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:26:43.936980 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'tag': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df', 'status': 'completed', 'code': 200} May 24 11:26:44.332969 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0cbcdfb-d083-4bb7-a7f7-7cd0f6cbcf9a req-573074fe-b0db-4cb8-9b4e-e2f5109a0f4a tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:44.334147 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-e0cbcdfb-d083-4bb7-a7f7-7cd0f6cbcf9a req-573074fe-b0db-4cb8-9b4e-e2f5109a0f4a tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Synchronizing usage tracker for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:26:44.360457 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-e0cbcdfb-d083-4bb7-a7f7-7cd0f6cbcf9a req-573074fe-b0db-4cb8-9b4e-e2f5109a0f4a tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Unset dirty status for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:26:44.368910 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e0cbcdfb-d083-4bb7-a7f7-7cd0f6cbcf9a req-573074fe-b0db-4cb8-9b4e-e2f5109a0f4a tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 945 time: 0.3303826 May 24 11:26:44.418475 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-46551a1c-5d6d-46d7-8e28-f55cea9ad787 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': 'bfd25b24-9190-4f5b-be56-6f7de757deb6', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.8', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'fixed_ip_address': '172.20.0.9', 'status': 'ACTIVE', 'standard_attr_id': 534, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:a6:bc:9e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '45596010-5877-4cfc-8808-752c422ae93a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:23:06Z', 'updated_at': '2021-05-24T11:23:09Z', 'revision_number': 1, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:26:44.426422 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-46551a1c-5d6d-46d7-8e28-f55cea9ad787 None None] Checking router: 81705dbf-841d-45b4-8b2d-afa9886a8e59 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:26:44.440700 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-46551a1c-5d6d-46d7-8e28-f55cea9ad787 None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:26:44.441056 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8e256b9-e50a-4338-9db4-c9f01106457c req-f290c90f-78ef-44e6-93c1-0c6ddb213a17 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:44.441575 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-46551a1c-5d6d-46d7-8e28-f55cea9ad787 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:26:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:26:44.443493 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8e256b9-e50a-4338-9db4-c9f01106457c req-f290c90f-78ef-44e6-93c1-0c6ddb213a17 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 1098 time: 0.4850535 May 24 11:26:44.600625 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e0cbcdfb-d083-4bb7-a7f7-7cd0f6cbcf9a req-94bcba42-f79c-42d5-8b7d-55d36f589a7a tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2299483 May 24 11:26:44.638044 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8e256b9-e50a-4338-9db4-c9f01106457c req-0d37a409-0a94-4db8-843f-9265edc109fa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:44.639757 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8e256b9-e50a-4338-9db4-c9f01106457c req-0d37a409-0a94-4db8-843f-9265edc109fa service neutron] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 912 time: 0.1869371 May 24 11:26:44.669158 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-46551a1c-5d6d-46d7-8e28-f55cea9ad787 None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:26:44.714590 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8e256b9-e50a-4338-9db4-c9f01106457c req-e67c7fa9-3550-4dba-a4ea-37a799ccbe2b service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=f7c7da3d-a1d5-44d2-a684-ba1255d063df HTTP/1.1" status: 200 len: 217 time: 0.0676906 May 24 11:26:44.779265 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8e256b9-e50a-4338-9db4-c9f01106457c req-52b8ba48-b1ed-4f3c-831c-b5072fa33085 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:44.780720 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8e256b9-e50a-4338-9db4-c9f01106457c req-52b8ba48-b1ed-4f3c-831c-b5072fa33085 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.0595100 May 24 11:26:44.902286 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8e256b9-e50a-4338-9db4-c9f01106457c req-5b8198ef-cb21-42cb-b183-00211e2361a0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:44.905706 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8e256b9-e50a-4338-9db4-c9f01106457c req-5b8198ef-cb21-42cb-b183-00211e2361a0 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1182017 May 24 11:26:45.045542 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8e256b9-e50a-4338-9db4-c9f01106457c req-0ae52b95-2da2-4581-ab3f-d98802c5f5cf service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1329286 May 24 11:26:45.128628 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-46551a1c-5d6d-46d7-8e28-f55cea9ad787 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': 'bfd25b24-9190-4f5b-be56-6f7de757deb6', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.8', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'fixed_ip_address': '172.20.0.9', 'status': 'ACTIVE', 'standard_attr_id': 534, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:a6:bc:9e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '45596010-5877-4cfc-8808-752c422ae93a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:23:06Z', 'updated_at': '2021-05-24T11:23:09Z', 'revision_number': 1, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:26:45.133904 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-46551a1c-5d6d-46d7-8e28-f55cea9ad787 None None] The sync data for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi is: [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': 'bfd25b24-9190-4f5b-be56-6f7de757deb6', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'floating_ip_address': '172.27.0.8', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'port_id': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'fixed_ip_address': '172.20.0.9', 'status': 'ACTIVE', 'standard_attr_id': 534, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'mac_address': 'fa:16:3e:a6:bc:9e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '45596010-5877-4cfc-8808-752c422ae93a', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:23:06Z', 'updated_at': '2021-05-24T11:23:09Z', 'revision_number': 1, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:26:45.187227 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f8e256b9-e50a-4338-9db4-c9f01106457c req-acf5d893-dfab-4d87-be90-6eef8a28472d service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1349397 May 24 11:26:45.370584 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-85ab5641-39e9-48d6-9728-7c7f4b42a25e None None] New status for floating IP f469c6c5-0e2c-4d4c-8124-71782bcea9ea: DOWN {{(pid=91822) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:26:45.580845 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38c9b195-3f41-44c3-b292-e5d299445a4c req-7246efc5-1963-4fe8-afda-297c6eef460b tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:45.582339 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-38c9b195-3f41-44c3-b292-e5d299445a4c req-7246efc5-1963-4fe8-afda-297c6eef460b tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 945 time: 0.1283469 May 24 11:26:45.804972 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-38c9b195-3f41-44c3-b292-e5d299445a4c req-c8bf525c-6fa4-4f7e-a9a2-017a1c1754b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Deleting port f7c7da3d-a1d5-44d2-a684-ba1255d063df {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:26:45.805399 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-38c9b195-3f41-44c3-b292-e5d299445a4c req-c8bf525c-6fa4-4f7e-a9a2-017a1c1754b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:46.095615 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-38c9b195-3f41-44c3-b292-e5d299445a4c req-c8bf525c-6fa4-4f7e-a9a2-017a1c1754b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] For port f7c7da3d-a1d5-44d2-a684-ba1255d063df, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=f7c7da3d-a1d5-44d2-a684-ba1255d063df,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:26:46.095961 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-38c9b195-3f41-44c3-b292-e5d299445a4c req-c8bf525c-6fa4-4f7e-a9a2-017a1c1754b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:46.168878 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f54e238-0842-4118-9af4-55a034380d38 req-3b4ff3d1-417f-4624-8ed0-3e7e248386b6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:46.170464 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1f54e238-0842-4118-9af4-55a034380d38 req-3b4ff3d1-417f-4624-8ed0-3e7e248386b6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=72051a16-fa49-4173-8c9c-ccbc49356cc3 HTTP/1.1" status: 200 len: 945 time: 0.3752789 May 24 11:26:46.229972 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-38c9b195-3f41-44c3-b292-e5d299445a4c req-c8bf525c-6fa4-4f7e-a9a2-017a1c1754b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Calling delete_port for f7c7da3d-a1d5-44d2-a684-ba1255d063df owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:26:46.284503 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1f54e238-0842-4118-9af4-55a034380d38 req-2a627cc7-a938-4205-ad24-fac64de3ca39 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0615933 May 24 11:26:46.378018 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-38c9b195-3f41-44c3-b292-e5d299445a4c req-c8bf525c-6fa4-4f7e-a9a2-017a1c1754b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:46.384050 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d7d1aa19-8aad-4e1c-ab8e-0c67192ff7ab admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:26:46.429685 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c8bf525c-6fa4-4f7e-a9a2-017a1c1754b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event deleted for resources: {'Port': ['ID=f7c7da3d-a1d5-44d2-a684-ba1255d063df,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:26:46.466410 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c8bf525c-6fa4-4f7e-a9a2-017a1c1754b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:26:46.507120 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-38c9b195-3f41-44c3-b292-e5d299445a4c req-c8bf525c-6fa4-4f7e-a9a2-017a1c1754b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] No DVR routers for this DVR port f7c7da3d-a1d5-44d2-a684-ba1255d063df on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:26:46.510502 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-38c9b195-3f41-44c3-b292-e5d299445a4c req-c8bf525c-6fa4-4f7e-a9a2-017a1c1754b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:46.513274 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'name': 'network-vif-deleted', 'tag': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:26:46.533030 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-38c9b195-3f41-44c3-b292-e5d299445a4c req-c8bf525c-6fa4-4f7e-a9a2-017a1c1754b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "DELETE /v2.0/ports/f7c7da3d-a1d5-44d2-a684-ba1255d063df HTTP/1.1" status: 204 len: 173 time: 0.9447403 May 24 11:26:46.625930 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '72051a16-fa49-4173-8c9c-ccbc49356cc3', 'name': 'network-vif-deleted', 'tag': 'f7c7da3d-a1d5-44d2-a684-ba1255d063df', 'status': 'completed', 'code': 200} May 24 11:26:48.295963 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:26:50.009245 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device f7c7da3d-a1d5-44d2-a684-ba1255d063df no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:26:50.048035 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: f7c7da3d-a1d5-44d2-a684-ba1255d063df {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:26:50.048389 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device f7c7da3d-a1d5-44d2-a684-ba1255d063df not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:26:51.911432 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33664) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:26:52.106468 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:52.187254 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Deleting port 332cf5a0-e4e4-4e30-be84-a0e470103da6 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:26:52.187602 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:52.307250 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:52.307570 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:52.429834 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Calling delete_port for 332cf5a0-e4e4-4e30-be84-a0e470103da6 owned by network:floatingip {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:26:52.558381 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:52.564445 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:26:52.609878 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event deleted for resources: {'Port': ['ID=332cf5a0-e4e4-4e30-be84-a0e470103da6,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:26:52.635694 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:26:52.702442 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:52.702801 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.l3_db [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Floating IP bfd25b24-9190-4f5b-be56-6f7de757deb6 disassociated (deleted). External IP: 172.27.0.8, port: 8ca4165e-4c84-4940-9148-3f5a77eaf348. May 24 11:26:53.070841 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Router 81705dbf-841d-45b4-8b2d-afa9886a8e59 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:26:53.260876 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:26:53.270530 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b45d38b9-c57b-43bf-aa2e-e4afa1c8c15e None None] Sync routers for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:26:53.276855 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:53.279823 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b45d38b9-c57b-43bf-aa2e-e4afa1c8c15e None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:26:53.280490 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-b45d38b9-c57b-43bf-aa2e-e4afa1c8c15e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:26:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:26:53.440186 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-056dc17b-d03b-4f42-b480-0184b44164d6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "DELETE /v2.0/floatingips/bfd25b24-9190-4f5b-be56-6f7de757deb6 HTTP/1.1" status: 204 len: 168 time: 1.5276756 May 24 11:26:53.441043 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '45596010-5877-4cfc-8808-752c422ae93a', 'tag': '8ca4165e-4c84-4940-9148-3f5a77eaf348'}] {{(pid=91820) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:26:53.443882 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f8b04b06-088c-4a36-ac4c-7c19710b1b52 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:53.476130 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-b45d38b9-c57b-43bf-aa2e-e4afa1c8c15e None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:26:53.487827 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '45596010-5877-4cfc-8808-752c422ae93a', 'tag': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'status': 'completed', 'code': 200} May 24 11:26:53.584262 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-600155fd-1e17-4783-af40-012a35b187ba req-4941e64e-6d3c-4666-a806-e057abe44359 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:53.585843 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-600155fd-1e17-4783-af40-012a35b187ba req-4941e64e-6d3c-4666-a806-e057abe44359 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 1098 time: 0.0746086 May 24 11:26:53.809676 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-b45d38b9-c57b-43bf-aa2e-e4afa1c8c15e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:26:53.813541 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b45d38b9-c57b-43bf-aa2e-e4afa1c8c15e None None] Checking router: 81705dbf-841d-45b4-8b2d-afa9886a8e59 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:26:53.821889 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b45d38b9-c57b-43bf-aa2e-e4afa1c8c15e None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:26:53.822500 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-b45d38b9-c57b-43bf-aa2e-e4afa1c8c15e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:26:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:26:53.883084 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-600155fd-1e17-4783-af40-012a35b187ba req-702d2d09-9621-476e-a35a-100d1a2765f1 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:53.884208 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-600155fd-1e17-4783-af40-012a35b187ba req-702d2d09-9621-476e-a35a-100d1a2765f1 service neutron] 10.222.0.93 "GET /v2.0/networks?id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 200 len: 912 time: 0.2912631 May 24 11:26:53.893288 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d4c0a19d-cd58-4d24-b293-057a72bf88f5 req-65d0266a-3c3f-493a-beb4-67ac043ad16f tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:53.893951 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-d4c0a19d-cd58-4d24-b293-057a72bf88f5 req-65d0266a-3c3f-493a-beb4-67ac043ad16f tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Synchronizing usage tracker for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:26:53.969756 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-600155fd-1e17-4783-af40-012a35b187ba req-23ca6497-f231-4a56-ad4a-6442b0cbe1bd service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=8ca4165e-4c84-4940-9148-3f5a77eaf348 HTTP/1.1" status: 200 len: 217 time: 0.0710084 May 24 11:26:54.009411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-d4c0a19d-cd58-4d24-b293-057a72bf88f5 req-65d0266a-3c3f-493a-beb4-67ac043ad16f tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Unset dirty status for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:26:54.010467 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d4c0a19d-cd58-4d24-b293-057a72bf88f5 req-65d0266a-3c3f-493a-beb4-67ac043ad16f tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 945 time: 0.2480574 May 24 11:26:54.023143 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-b45d38b9-c57b-43bf-aa2e-e4afa1c8c15e None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:26:54.064249 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-600155fd-1e17-4783-af40-012a35b187ba req-a58377a9-ec7f-42b8-96aa-e8e82110a90b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:54.065791 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-600155fd-1e17-4783-af40-012a35b187ba req-a58377a9-ec7f-42b8-96aa-e8e82110a90b service neutron] 10.222.0.93 "GET /v2.0/subnets?id=994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 200 len: 850 time: 0.0899019 May 24 11:26:54.077353 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d4c0a19d-cd58-4d24-b293-057a72bf88f5 req-57626b7f-9a73-4b97-a7fa-d92a384d469c tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0586932 May 24 11:26:54.158686 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-600155fd-1e17-4783-af40-012a35b187ba req-9ec5d5dc-2e70-4087-9047-e33154001881 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:54.159604 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-600155fd-1e17-4783-af40-012a35b187ba req-9ec5d5dc-2e70-4087-9047-e33154001881 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=be65d1af-351e-4e5b-8c19-1b04ff5cfc29&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0855675 May 24 11:26:54.305553 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-600155fd-1e17-4783-af40-012a35b187ba req-bc7541fd-5e5d-4a31-9be5-e6766e70e186 service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1381276 May 24 11:26:54.397588 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-b45d38b9-c57b-43bf-aa2e-e4afa1c8c15e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:26:54.403314 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b45d38b9-c57b-43bf-aa2e-e4afa1c8c15e None None] The sync data for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi is: [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:18:58Z', 'revision_number': 4, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '5646b226-211d-4c06-b7f8-5dfc132e817e', 'name': '', 'network_id': 'be65d1af-351e-4e5b-8c19-1b04ff5cfc29', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'mac_address': 'fa:16:3e:88:ae:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_interface', 'standard_attr_id': 391, 'fixed_ips': [{'subnet_id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:56Z', 'updated_at': '2021-05-24T11:19:11Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'subnets': [{'id': '994391be-baae-4f60-9ef9-bf02d19d3477', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:26:54.449628 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-600155fd-1e17-4783-af40-012a35b187ba req-a0e8e3cb-f22f-4217-8cf0-a44fcfd0211a service neutron] 10.222.0.93 "GET /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1346378 May 24 11:26:54.612151 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-859b55bc-337a-4f6f-be96-292d38f376b0 None None] New status for floating IP bfd25b24-9190-4f5b-be56-6f7de757deb6: DOWN {{(pid=91822) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:26:55.207281 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b88e2403-29d8-4579-9840-e037b728748b req-c0a7c651-dfdc-4bf8-b306-bb1b050b9473 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:55.208685 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b88e2403-29d8-4579-9840-e037b728748b req-c0a7c651-dfdc-4bf8-b306-bb1b050b9473 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 945 time: 0.0789616 May 24 11:26:55.276017 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ae69b73-3b75-48b2-8141-78be64ed1233 req-c397cc16-b8e1-41e9-b72a-4a8948232b7e tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:26:55.277009 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-0ae69b73-3b75-48b2-8141-78be64ed1233 req-c397cc16-b8e1-41e9-b72a-4a8948232b7e tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Synchronizing usage tracker for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:26:55.288959 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-0ae69b73-3b75-48b2-8141-78be64ed1233 req-c397cc16-b8e1-41e9-b72a-4a8948232b7e tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Unset dirty status for tenant:9a8d61f79bbc496ca30e390fd9b5839e on resource:port {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:26:55.290168 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0ae69b73-3b75-48b2-8141-78be64ed1233 req-c397cc16-b8e1-41e9-b72a-4a8948232b7e tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 945 time: 0.0935624 May 24 11:26:55.343172 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0ae69b73-3b75-48b2-8141-78be64ed1233 req-a67a5d9c-bb8a-4ef6-9f40-5d37950831ba tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/security-groups?id=5928fed2-3207-44a0-91e4-485200bd1a27&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0464048 May 24 11:26:55.350714 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-b88e2403-29d8-4579-9840-e037b728748b req-1060cae6-53b2-4620-9a7d-090c0c434d44 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Deleting port 8ca4165e-4c84-4940-9148-3f5a77eaf348 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:26:55.351185 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-b88e2403-29d8-4579-9840-e037b728748b req-1060cae6-53b2-4620-9a7d-090c0c434d44 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:55.507559 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-b88e2403-29d8-4579-9840-e037b728748b req-1060cae6-53b2-4620-9a7d-090c0c434d44 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] For port 8ca4165e-4c84-4940-9148-3f5a77eaf348, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=8ca4165e-4c84-4940-9148-3f5a77eaf348,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:26:55.507917 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-b88e2403-29d8-4579-9840-e037b728748b req-1060cae6-53b2-4620-9a7d-090c0c434d44 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:55.644259 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-b88e2403-29d8-4579-9840-e037b728748b req-1060cae6-53b2-4620-9a7d-090c0c434d44 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Calling delete_port for 8ca4165e-4c84-4940-9148-3f5a77eaf348 owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:26:55.755918 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-b88e2403-29d8-4579-9840-e037b728748b req-1060cae6-53b2-4620-9a7d-090c0c434d44 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:55.762802 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-79e275b4-6fd7-4209-b639-1d8abe49f056 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:26:55.763021 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:55.807927 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1060cae6-53b2-4620-9a7d-090c0c434d44 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event deleted for resources: {'Port': ['ID=8ca4165e-4c84-4940-9148-3f5a77eaf348,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:26:55.811675 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-1060cae6-53b2-4620-9a7d-090c0c434d44 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:26:55.887153 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-b88e2403-29d8-4579-9840-e037b728748b req-1060cae6-53b2-4620-9a7d-090c0c434d44 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] No DVR routers for this DVR port 8ca4165e-4c84-4940-9148-3f5a77eaf348 on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:26:55.889542 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8c85512a-0e1c-482f-a229-4256b3d155a6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:55.890151 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-b88e2403-29d8-4579-9840-e037b728748b req-1060cae6-53b2-4620-9a7d-090c0c434d44 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:55.892925 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '45596010-5877-4cfc-8808-752c422ae93a', 'name': 'network-vif-deleted', 'tag': '8ca4165e-4c84-4940-9148-3f5a77eaf348'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:26:55.910072 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b88e2403-29d8-4579-9840-e037b728748b req-1060cae6-53b2-4620-9a7d-090c0c434d44 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.93 "DELETE /v2.0/ports/8ca4165e-4c84-4940-9148-3f5a77eaf348 HTTP/1.1" status: 204 len: 173 time: 0.6950352 May 24 11:26:55.952598 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '45596010-5877-4cfc-8808-752c422ae93a', 'name': 'network-vif-deleted', 'tag': '8ca4165e-4c84-4940-9148-3f5a77eaf348', 'status': 'completed', 'code': 200} May 24 11:26:56.506990 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6f0b846f-a590-475c-9cd6-a5041eaa3ba4 req-8994232d-364e-462c-8b80-53163acb69b9 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "GET /v2.0/ports?device_id=45596010-5877-4cfc-8808-752c422ae93a HTTP/1.1" status: 200 len: 210 time: 0.0414853 May 24 11:26:57.005254 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:57.603700 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33778) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:26:57.981719 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-e28d571c-aeb2-49fa-9cfd-a6713096cbbb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Deleting port 0cf9a14b-892a-4235-8708-1f72096a9a29 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:26:57.982064 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e28d571c-aeb2-49fa-9cfd-a6713096cbbb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:58.122609 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e28d571c-aeb2-49fa-9cfd-a6713096cbbb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:58.122981 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e28d571c-aeb2-49fa-9cfd-a6713096cbbb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:58.272661 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-e28d571c-aeb2-49fa-9cfd-a6713096cbbb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Calling delete_port for 0cf9a14b-892a-4235-8708-1f72096a9a29 owned by network:floatingip {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:26:58.384328 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e28d571c-aeb2-49fa-9cfd-a6713096cbbb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:58.388620 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9f9e31c4-444f-434c-bee3-9b1338a3b565 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:26:58.436476 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e28d571c-aeb2-49fa-9cfd-a6713096cbbb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event deleted for resources: {'Port': ['ID=0cf9a14b-892a-4235-8708-1f72096a9a29,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:26:58.462724 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e28d571c-aeb2-49fa-9cfd-a6713096cbbb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:26:58.529887 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e28d571c-aeb2-49fa-9cfd-a6713096cbbb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for floatingip, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:58.543653 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e28d571c-aeb2-49fa-9cfd-a6713096cbbb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:58.544892 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e28d571c-aeb2-49fa-9cfd-a6713096cbbb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "DELETE /v2.0/floatingips/6caa87da-7c20-4c7c-a70d-c42930870a74 HTTP/1.1" status: 204 len: 168 time: 0.9400520 May 24 11:26:58.946965 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:26:59.920752 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 8ca4165e-4c84-4940-9148-3f5a77eaf348 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:26:59.927812 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33796) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:26:59.955176 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 8ca4165e-4c84-4940-9148-3f5a77eaf348 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:26:59.955762 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 8ca4165e-4c84-4940-9148-3f5a77eaf348 not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:27:00.002833 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:00.035553 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:00.106909 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:00.111173 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a99bb5c0-93ff-4d68-bf54-46022fa46a62 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:00.124927 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=5928fed2-3207-44a0-91e4-485200bd1a27,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:00.134089 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:00.134381 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:00.134746 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:00.135146 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:00.135579 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:00.135945 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:00.136308 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:00.136800 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:00.137118 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:00.137471 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:00.137855 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:00.138170 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:00.143225 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:00.146752 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:00.150643 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] 10.222.0.18 "DELETE /v2.0/security-groups/5928fed2-3207-44a0-91e4-485200bd1a27 HTTP/1.1" status: 204 len: 168 time: 0.2190590 May 24 11:27:00.206526 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33802) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:00.876756 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-687465'] for router_interface, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:01.300042 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Deleting port 5646b226-211d-4c06-b7f8-5dfc132e817e {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:27:01.300712 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:01.425638 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] For port 5646b226-211d-4c06-b7f8-5dfc132e817e, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=5646b226-211d-4c06-b7f8-5dfc132e817e,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:27:01.426031 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:01.664884 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Calling delete_port for 5646b226-211d-4c06-b7f8-5dfc132e817e owned by network:router_interface {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:27:01.810757 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:01.814281 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-80c13359-b3e4-4143-95f7-de818947dcc0 tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:01.861155 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Pushing event deleted for resources: {'Port': ['ID=5646b226-211d-4c06-b7f8-5dfc132e817e,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:01.905139 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:01.905726 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:01.905896 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:01.906254 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:01.906609 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:01.906967 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:01.907316 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:01.907664 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:01.908014 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:01.908361 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:01.908706 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:01.909051 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:01.939566 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.125s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:02.140459 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-537767', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-685561'] for router_interface, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:02.503360 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Router 81705dbf-841d-45b4-8b2d-afa9886a8e59 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:27:02.682299 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:27:02.691965 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8cd04d46-fe8c-4958-a214-033931015732 None None] Sync routers for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:27:02.697129 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-a8aa3120-fd44-40ee-a7f3-1affa8cc484a admin admin] 10.222.0.18 "PUT /v2.0/routers/81705dbf-841d-45b4-8b2d-afa9886a8e59/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.4877160 May 24 11:27:02.697841 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33814) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:02.703226 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8cd04d46-fe8c-4958-a214-033931015732 None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:27:02.703918 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-8cd04d46-fe8c-4958-a214-033931015732 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:26:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:02.873566 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Notify callbacks [] for router, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:02.888269 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-8cd04d46-fe8c-4958-a214-033931015732 None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:27:03.171252 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:03.188359 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-8cd04d46-fe8c-4958-a214-033931015732 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29'}], 'enable_snat': True}, 'gw_port_id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:27:01Z', 'revision_number': 5, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port': {'id': 'e648ddfe-6f08-4b5c-8c21-e6794aa56080', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:8e:d7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'device_owner': 'network:router_gateway', 'standard_attr_id': 389, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.29', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:18:50Z', 'updated_at': '2021-05-24T11:19:03Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:03.192190 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8cd04d46-fe8c-4958-a214-033931015732 None None] Checking router: 81705dbf-841d-45b4-8b2d-afa9886a8e59 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:27:03.200572 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8cd04d46-fe8c-4958-a214-033931015732 None None] Routers {'81705dbf-841d-45b4-8b2d-afa9886a8e59'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:27:03.201220 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-8cd04d46-fe8c-4958-a214-033931015732 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:26:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['81705dbf-841d-45b4-8b2d-afa9886a8e59'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:03.242075 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Deleting port e648ddfe-6f08-4b5c-8c21-e6794aa56080 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:27:03.242409 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:03.353781 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-8cd04d46-fe8c-4958-a214-033931015732 None None] Set router 81705dbf-841d-45b4-8b2d-afa9886a8e59 gateway port host: None {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:27:03.367906 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] For port e648ddfe-6f08-4b5c-8c21-e6794aa56080, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=e648ddfe-6f08-4b5c-8c21-e6794aa56080,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:27:03.368289 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:03.458791 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-8cd04d46-fe8c-4958-a214-033931015732 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:27:03Z', 'revision_number': 6, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port_host': None}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:03.462725 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8cd04d46-fe8c-4958-a214-033931015732 None None] The sync data for ids ['81705dbf-841d-45b4-8b2d-afa9886a8e59'] in dv-cinder-784092-9-iscsi is: [{'id': '81705dbf-841d-45b4-8b2d-afa9886a8e59', 'name': 'tempest-TestVolumeBootPattern-908540324-router', 'tenant_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:18:48Z', 'updated_at': '2021-05-24T11:27:03Z', 'revision_number': 6, 'project_id': '9a8d61f79bbc496ca30e390fd9b5839e', 'gw_port_host': None}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:27:03.505186 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Calling delete_port for e648ddfe-6f08-4b5c-8c21-e6794aa56080 owned by network:router_gateway {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:27:03.651007 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:03.654595 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:03.700254 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Pushing event deleted for resources: {'Port': ['ID=e648ddfe-6f08-4b5c-8c21-e6794aa56080,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:03.703391 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:03.795824 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-686964'] for router_gateway, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:03.972466 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-684537', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4171'] for router, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:03.974626 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:03.998090 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-684229'] for router, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:03.998549 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 81705dbf-841d-45b4-8b2d-afa9886a8e59 {{(pid=91820) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 11:27:04.010551 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Notify callbacks [] for router, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:04.012375 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] 10.222.0.18 "DELETE /v2.0/routers/81705dbf-841d-45b4-8b2d-afa9886a8e59 HTTP/1.1" status: 204 len: 168 time: 1.3135986 May 24 11:27:04.015110 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33820) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:04.077661 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_v2 [None req-3bc706f6-38b1-4e63-9699-f135ea5d1341 admin admin] Deleting subnet 994391be-baae-4f60-9ef9-bf02d19d3477 {{(pid=91820) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:27:04.080642 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3bc706f6-38b1-4e63-9699-f135ea5d1341 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:04.137697 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3bc706f6-38b1-4e63-9699-f135ea5d1341 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534861947'] for subnet, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:04.218869 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3bc706f6-38b1-4e63-9699-f135ea5d1341 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:04.479186 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3bc706f6-38b1-4e63-9699-f135ea5d1341 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534788488', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535456750'] for subnet, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:04.485747 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:04.520516 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3bc706f6-38b1-4e63-9699-f135ea5d1341 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=994391be-baae-4f60-9ef9-bf02d19d3477,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:04.523510 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-3bc706f6-38b1-4e63-9699-f135ea5d1341 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:04.527832 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-3bc706f6-38b1-4e63-9699-f135ea5d1341 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:04.533122 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-815cdb81-cf69-4b60-ac79-e0ca540b2840 None None] Network be65d1af-351e-4e5b-8c19-1b04ff5cfc29 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:27:04.534179 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-3bc706f6-38b1-4e63-9699-f135ea5d1341 admin admin] 10.222.0.18 "DELETE /v2.0/subnets/994391be-baae-4f60-9ef9-bf02d19d3477 HTTP/1.1" status: 204 len: 168 time: 0.5180194 May 24 11:27:04.536476 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33824) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:04.712021 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device e648ddfe-6f08-4b5c-8c21-e6794aa56080 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:27:04.752416 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: e648ddfe-6f08-4b5c-8c21-e6794aa56080 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:27:04.752768 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device e648ddfe-6f08-4b5c-8c21-e6794aa56080 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:27:04.753587 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 5646b226-211d-4c06-b7f8-5dfc132e817e no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:27:04.774766 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:04.789130 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 5646b226-211d-4c06-b7f8-5dfc132e817e {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:27:04.789130 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 5646b226-211d-4c06-b7f8-5dfc132e817e not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:27:04.852429 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Deleting port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:27:04.852768 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:04.892651 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] DHCP port deletion for be65d1af-351e-4e5b-8c19-1b04ff5cfc29 request from dv-cinder-784092-9-iscsi {{(pid=91821) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 11:27:04.955471 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] For port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=4e43bacd-47d7-4c30-b3b3-64bc8458a6dc,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:27:04.955823 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:04.963927 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Deleting port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc {{(pid=91821) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:27:04.964335 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:05.082100 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Calling delete_port for 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc owned by network:dhcp {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:27:05.105244 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] For port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=4e43bacd-47d7-4c30-b3b3-64bc8458a6dc,segment=NetworkSegment(2f7d3253-3331-43a2-800a-8858592fd924),segment_id=2f7d3253-3331-43a2-800a-8858592fd924)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:27:05.105634 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:05.136562 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:05.140317 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-f9c497d6-bac8-4a16-8176-3f304e25b916 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:05.185273 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Pushing event deleted for resources: {'Port': ['ID=4e43bacd-47d7-4c30-b3b3-64bc8458a6dc,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:05.188340 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:05.189126 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] No DVR routers for this DVR port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc on host dv-cinder-784092-9-iscsi {{(pid=91820) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:27:05.230064 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Calling delete_port for 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc owned by network:dhcp {{(pid=91821) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:27:05.273442 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093'] for port, after_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:05.277551 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:05.310173 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:05.322868 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['be65d1af-351e-4e5b-8c19-1b04ff5cfc29']}} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:05.325283 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Pushing event deleted for resources: {'Port': ['ID=4e43bacd-47d7-4c30-b3b3-64bc8458a6dc,revision_number=None']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:05.332596 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] No DVR routers for this DVR port 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc on host dv-cinder-784092-9-iscsi {{(pid=91821) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:27:05.336344 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '2f7d3253-3331-43a2-800a-8858592fd924') {'for_net_delete': True} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:05.336646 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '2f7d3253-3331-43a2-800a-8858592fd924') {} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:05.342648 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:05.343003 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:05.343502 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:05.343756 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:05.344116 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:05.344492 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:05.344865 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:05.345265 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:05.345773 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:05.346108 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:05.346476 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:05.346801 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:27:05.350262 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-04fb93ab-26c2-430d-bce3-1fa0b4a14895 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:05.356526 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534861936'] for segment, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:05.373443 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:05.379682 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Releasing vlan 503 on physical network physnet1 to pool {{(pid=91820) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:27:05.380066 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535262766'] for segment, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: Traceback (most recent call last): May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: timer() May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: cb(*args, **kw) May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: return func(*args, **kwargs) May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: self._notify() May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: self.callback(batched_events) May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: event.method(event) May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: return f(self, *a, **k) May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._get(url).json() May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: return self.request(url, 'GET', **kwargs) May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:27:05.393556 dv-cinder-784092-9-iscsi neutron-server[91820]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:27:05.472449 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:05.476366 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-1a209e56-2bca-465b-b611-7d28c690f088 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:05.489319 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:05.500958 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Pushing event deleted for resources: {'Network': ['ID=be65d1af-351e-4e5b-8c19-1b04ff5cfc29,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:05.507740 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:05.514713 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-645a0de3-ebca-4e3a-8258-56a99f5810fa admin admin] 10.222.0.18 "DELETE /v2.0/networks/be65d1af-351e-4e5b-8c19-1b04ff5cfc29 HTTP/1.1" status: 204 len: 168 time: 0.9768534 May 24 11:27:05.517503 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33830) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:06.058420 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-687465'] for router_interface, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:06.586391 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] Deleting port cbcf156d-db06-40df-9ad0-c2981b742561 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:27:06.586721 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:06.702381 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:27:06.710695 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] For port cbcf156d-db06-40df-9ad0-c2981b742561, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=cbcf156d-db06-40df-9ad0-c2981b742561,segment=NetworkSegment(c1fbdc79-842c-49eb-8789-2a1de3ab2b9f),segment_id=c1fbdc79-842c-49eb-8789-2a1de3ab2b9f)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:27:06.711046 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:06.740194 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:27:06.740546 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 4e43bacd-47d7-4c30-b3b3-64bc8458a6dc not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:27:06.833910 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] Calling delete_port for cbcf156d-db06-40df-9ad0-c2981b742561 owned by network:router_interface {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:27:06.978899 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:06.984395 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-6eddd231-0048-4e88-9668-aa2b252435fa tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:07.029747 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] Pushing event deleted for resources: {'Port': ['ID=cbcf156d-db06-40df-9ad0-c2981b742561,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:07.057124 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:07.297130 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-537767', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-685561'] for router_interface, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:07.656393 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] Router 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91820) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:27:07.938643 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:27:07.946910 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fdf64c14-ea2b-4cea-a303-a1237952ef6e None None] Sync routers for ids ['7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:27:07.954512 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-34c44fe2-e3f8-46fb-8547-727903bce4fb admin admin] 10.222.0.18 "PUT /v2.0/routers/7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.4359920 May 24 11:27:07.956926 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33842) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:07.957607 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fdf64c14-ea2b-4cea-a303-a1237952ef6e None None] Routers {'7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:27:07.958385 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-fdf64c14-ea2b-4cea-a303-a1237952ef6e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:26:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:08.134690 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Notify callbacks [] for router, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:08.145432 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-fdf64c14-ea2b-4cea-a303-a1237952ef6e None None] Set router 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:27:08.316140 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:08.386866 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Deleting port a4cc0b23-ca4a-4a18-9cee-ce596875baf8 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:27:08.387210 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:08.431334 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-fdf64c14-ea2b-4cea-a303-a1237952ef6e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'name': 'tempest-TestVolumeBootPattern-1369558850-router', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205'}], 'enable_snat': True}, 'gw_port_id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:19:08Z', 'updated_at': '2021-05-24T11:27:06Z', 'revision_number': 5, 'project_id': '672c5f4985084a8492a86667aceb0768', 'gw_port': {'id': 'a4cc0b23-ca4a-4a18-9cee-ce596875baf8', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:68:b5:40', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 403, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.205', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:19:10Z', 'updated_at': '2021-05-24T11:19:21Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:08.435137 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fdf64c14-ea2b-4cea-a303-a1237952ef6e None None] Checking router: 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:27:08.443398 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fdf64c14-ea2b-4cea-a303-a1237952ef6e None None] Routers {'7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:27:08.443998 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-fdf64c14-ea2b-4cea-a303-a1237952ef6e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:26:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:08.508452 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] For port a4cc0b23-ca4a-4a18-9cee-ce596875baf8, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=a4cc0b23-ca4a-4a18-9cee-ce596875baf8,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:27:08.508808 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:08.601036 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-fdf64c14-ea2b-4cea-a303-a1237952ef6e None None] Set router 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e gateway port host: None {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:27:08.634177 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Calling delete_port for a4cc0b23-ca4a-4a18-9cee-ce596875baf8 owned by network:router_gateway {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:27:08.738241 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-fdf64c14-ea2b-4cea-a303-a1237952ef6e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'name': 'tempest-TestVolumeBootPattern-1369558850-router', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:19:08Z', 'updated_at': '2021-05-24T11:27:08Z', 'revision_number': 6, 'project_id': '672c5f4985084a8492a86667aceb0768', 'gw_port_host': None}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:08.742249 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-fdf64c14-ea2b-4cea-a303-a1237952ef6e None None] The sync data for ids ['7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e'] in dv-cinder-784092-9-iscsi is: [{'id': '7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e', 'name': 'tempest-TestVolumeBootPattern-1369558850-router', 'tenant_id': '672c5f4985084a8492a86667aceb0768', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:19:08Z', 'updated_at': '2021-05-24T11:27:08Z', 'revision_number': 6, 'project_id': '672c5f4985084a8492a86667aceb0768', 'gw_port_host': None}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:27:08.783776 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:08.788547 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-265cf044-2581-42f8-92ee-e9baff16e18d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:08.857018 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Pushing event deleted for resources: {'Port': ['ID=a4cc0b23-ca4a-4a18-9cee-ce596875baf8,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:08.866570 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:08.923419 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-686964'] for router_gateway, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:09.216769 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-684537', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4171'] for router, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:09.218957 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:09.241500 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-684229'] for router, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:09.241927 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Fanout notify agent at l3_agent the message router_deleted on router 7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e {{(pid=91820) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 11:27:09.254014 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] Notify callbacks [] for router, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:09.255122 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-22b385ea-1c46-4303-b5ac-c6ea21009b9d admin admin] 10.222.0.18 "DELETE /v2.0/routers/7e0400a5-8b65-49ea-9a3e-80fc6fd0ec2e HTTP/1.1" status: 204 len: 168 time: 1.2972546 May 24 11:27:09.257582 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33858) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:09.318920 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_v2 [None req-a84ed81a-3b3a-4bcb-b184-ff89afda5b1a admin admin] Deleting subnet a25edff7-c0c6-44d4-ad1c-a38e28a26a35 {{(pid=91820) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:27:09.322229 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a84ed81a-3b3a-4bcb-b184-ff89afda5b1a admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:09.380108 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a84ed81a-3b3a-4bcb-b184-ff89afda5b1a admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534861947'] for subnet, precommit_delete_associations {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:09.463095 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a84ed81a-3b3a-4bcb-b184-ff89afda5b1a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:09.733642 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a84ed81a-3b3a-4bcb-b184-ff89afda5b1a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534788488', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535456750'] for subnet, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:09.737502 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:09.770927 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a84ed81a-3b3a-4bcb-b184-ff89afda5b1a admin admin] Pushing event deleted for resources: {'Subnet': ['ID=a25edff7-c0c6-44d4-ad1c-a38e28a26a35,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:09.773763 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-a84ed81a-3b3a-4bcb-b184-ff89afda5b1a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.036s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:09.778830 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-a84ed81a-3b3a-4bcb-b184-ff89afda5b1a admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:09.783366 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d2771059-e1d1-42e1-b6f1-70089846d461 None None] Network e6b6c00d-d044-4bc7-92ce-02e445f1f072 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:27:09.787354 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-a84ed81a-3b3a-4bcb-b184-ff89afda5b1a admin admin] 10.222.0.18 "DELETE /v2.0/subnets/a25edff7-c0c6-44d4-ad1c-a38e28a26a35 HTTP/1.1" status: 204 len: 168 time: 0.5282021 May 24 11:27:09.789359 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33862) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:09.899484 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:09.973895 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Deleting port 74b6164f-9aab-4c25-bfee-4839f99ffe91 {{(pid=91820) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:27:09.974164 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.025787 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-f13c6e44-2fe0-409a-971d-12b002fd70b8 None None] DHCP port deletion for e6b6c00d-d044-4bc7-92ce-02e445f1f072 request from dv-cinder-784092-9-iscsi {{(pid=91821) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 11:27:10.077748 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.db [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] For port 74b6164f-9aab-4c25-bfee-4839f99ffe91, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=74b6164f-9aab-4c25-bfee-4839f99ffe91,segment=NetworkSegment(c1fbdc79-842c-49eb-8789-2a1de3ab2b9f),segment_id=c1fbdc79-842c-49eb-8789-2a1de3ab2b9f)] {{(pid=91820) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:27:10.078087 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.093125 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-f13c6e44-2fe0-409a-971d-12b002fd70b8 None None] Deleting port 74b6164f-9aab-4c25-bfee-4839f99ffe91 {{(pid=91821) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:27:10.093506 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f13c6e44-2fe0-409a-971d-12b002fd70b8 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.192625 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-f13c6e44-2fe0-409a-971d-12b002fd70b8 None None] For port 74b6164f-9aab-4c25-bfee-4839f99ffe91, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=74b6164f-9aab-4c25-bfee-4839f99ffe91,segment=NetworkSegment(c1fbdc79-842c-49eb-8789-2a1de3ab2b9f),segment_id=c1fbdc79-842c-49eb-8789-2a1de3ab2b9f)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:27:10.193015 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f13c6e44-2fe0-409a-971d-12b002fd70b8 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.206383 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.plugin [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Calling delete_port for 74b6164f-9aab-4c25-bfee-4839f99ffe91 owned by network:dhcp {{(pid=91820) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:27:10.261415 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.265449 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-46e638dd-ecca-4cb8-90da-57f5e98bd069 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:10.311476 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Pushing event deleted for resources: {'Port': ['ID=74b6164f-9aab-4c25-bfee-4839f99ffe91,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:10.316571 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:10.322137 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] No DVR routers for this DVR port 74b6164f-9aab-4c25-bfee-4839f99ffe91 on host dv-cinder-784092-9-iscsi {{(pid=91820) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:27:10.336291 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-f13c6e44-2fe0-409a-971d-12b002fd70b8 None None] Calling delete_port for 74b6164f-9aab-4c25-bfee-4839f99ffe91 owned by network:dhcp {{(pid=91821) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:27:10.385735 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-f13c6e44-2fe0-409a-971d-12b002fd70b8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093'] for port, after_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.395520 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-90bd5ee8-51c8-4080-b0fe-668db0e19ac8 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:10.435335 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f13c6e44-2fe0-409a-971d-12b002fd70b8 None None] No DVR routers for this DVR port 74b6164f-9aab-4c25-bfee-4839f99ffe91 on host dv-cinder-784092-9-iscsi {{(pid=91821) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:27:10.446216 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f13c6e44-2fe0-409a-971d-12b002fd70b8 None None] Pushing event deleted for resources: {'Port': ['ID=74b6164f-9aab-4c25-bfee-4839f99ffe91,revision_number=None']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:10.451977 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.454310 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-f13c6e44-2fe0-409a-971d-12b002fd70b8 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:10.466216 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['e6b6c00d-d044-4bc7-92ce-02e445f1f072']}} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:10.478816 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'c1fbdc79-842c-49eb-8789-2a1de3ab2b9f') {'for_net_delete': True} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:10.479144 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.services.segments.db [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'c1fbdc79-842c-49eb-8789-2a1de3ab2b9f') {} {{(pid=91820) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:27:10.490646 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534861936'] for segment, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.507644 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.515549 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Releasing vlan 520 on physical network physnet1 to pool {{(pid=91820) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:27:10.516214 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535262766'] for segment, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: Traceback (most recent call last): May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: timer() May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: cb(*args, **kw) May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: return func(*args, **kwargs) May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: self._notify() May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: self.callback(batched_events) May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: event.method(event) May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: return f(self, *a, **k) May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._get(url).json() May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: return self.request(url, 'GET', **kwargs) May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:27:10.531105 dv-cinder-784092-9-iscsi neutron-server[91820]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:27:10.616802 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.621463 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:10.635325 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.645067 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Pushing event deleted for resources: {'Network': ['ID=e6b6c00d-d044-4bc7-92ce-02e445f1f072,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:10.649374 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:10.655772 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-2f437dcc-1c3e-4067-8d27-9d5b14c49e70 admin admin] 10.222.0.18 "DELETE /v2.0/networks/e6b6c00d-d044-4bc7-92ce-02e445f1f072 HTTP/1.1" status: 204 len: 168 time: 0.8627126 May 24 11:27:10.709075 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device cbcf156d-db06-40df-9ad0-c2981b742561 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:27:10.745783 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: cbcf156d-db06-40df-9ad0-c2981b742561 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:27:10.746172 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device cbcf156d-db06-40df-9ad0-c2981b742561 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:27:10.746612 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 74b6164f-9aab-4c25-bfee-4839f99ffe91 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:27:10.776900 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33872) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:10.783286 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 74b6164f-9aab-4c25-bfee-4839f99ffe91 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:27:10.783644 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 74b6164f-9aab-4c25-bfee-4839f99ffe91 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:27:10.784232 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device a4cc0b23-ca4a-4a18-9cee-ce596875baf8 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:27:10.817427 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: a4cc0b23-ca4a-4a18-9cee-ce596875baf8 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:27:10.817723 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device a4cc0b23-ca4a-4a18-9cee-ce596875baf8 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:27:10.829738 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-ab51ae5f-6fb5-4b3c-ab53-414e6b206654 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=9a8d61f79bbc496ca30e390fd9b5839e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0513251 May 24 11:27:10.832096 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33876) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:10.895611 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2ed2fc6a-71d1-4d29-8b0a-b2b4ce926fe7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.921931 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2ed2fc6a-71d1-4d29-8b0a-b2b4ce926fe7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.973840 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2ed2fc6a-71d1-4d29-8b0a-b2b4ce926fe7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:10.977470 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-e1591872-ec09-4928-bf0e-bf973af49d07 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:10.991973 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2ed2fc6a-71d1-4d29-8b0a-b2b4ce926fe7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e5fa59cc-2474-401d-98cd-11d70d470d79,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:10.996089 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-2ed2fc6a-71d1-4d29-8b0a-b2b4ce926fe7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:10.999301 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2ed2fc6a-71d1-4d29-8b0a-b2b4ce926fe7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:11.002715 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-2ed2fc6a-71d1-4d29-8b0a-b2b4ce926fe7 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/e5fa59cc-2474-401d-98cd-11d70d470d79 HTTP/1.1" status: 204 len: 168 time: 0.1674318 May 24 11:27:11.344135 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33886) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:11.404791 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-3b323942-8a96-4bf4-b4cd-eed2f2336248 admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=672c5f4985084a8492a86667aceb0768&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0591083 May 24 11:27:11.406990 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33890) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:11.470984 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5a09a858-935c-4e0b-a6c0-fd36af247cd8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:11.496405 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5a09a858-935c-4e0b-a6c0-fd36af247cd8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:11.553291 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5a09a858-935c-4e0b-a6c0-fd36af247cd8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:11.557705 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-fa9bb883-0544-46e5-903c-40bb4cb0cfc1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:11.568382 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5a09a858-935c-4e0b-a6c0-fd36af247cd8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=160a5414-06db-4f2b-92ba-2a3a3924021e,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:11.572139 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-5a09a858-935c-4e0b-a6c0-fd36af247cd8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:11.575920 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-5a09a858-935c-4e0b-a6c0-fd36af247cd8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:11.578432 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-5a09a858-935c-4e0b-a6c0-fd36af247cd8 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/160a5414-06db-4f2b-92ba-2a3a3924021e HTTP/1.1" status: 204 len: 168 time: 0.1691365 May 24 11:27:18.486608 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33940) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:18.622888 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e3d507e3-64aa-42e4-823e-5ebd21c2971f admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:18.643037 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e3d507e3-64aa-42e4-823e-5ebd21c2971f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:27:18.648594 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e3d507e3-64aa-42e4-823e-5ebd21c2971f admin admin] Usage tracker for resource:security_group_rule and tenant:3c020271143d4609bd71a98a7093af94 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:27:18.653527 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e3d507e3-64aa-42e4-823e-5ebd21c2971f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:27:18.660654 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e3d507e3-64aa-42e4-823e-5ebd21c2971f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:27:18.751464 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e3d507e3-64aa-42e4-823e-5ebd21c2971f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:18.756807 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e3d507e3-64aa-42e4-823e-5ebd21c2971f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:18.766893 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ff39643d-b6e1-45fc-959c-542e3afc67c2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:18.815903 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e3d507e3-64aa-42e4-823e-5ebd21c2971f admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=3c020271143d4609bd71a98a7093af94&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3281012 May 24 11:27:18.816828 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e3d507e3-64aa-42e4-823e-5ebd21c2971f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1a4fd47d-f2c0-407d-9b72-65fe77fadf08,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:18.818884 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33946) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:18.820099 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e3d507e3-64aa-42e4-823e-5ebd21c2971f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:18.880487 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-7cef49f6-601e-49a2-85a5-7d5d713d1f15 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:18.913556 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-7cef49f6-601e-49a2-85a5-7d5d713d1f15 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:18.987925 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-7cef49f6-601e-49a2-85a5-7d5d713d1f15 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:18.999579 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0bcf5526-b201-4e36-89f8-411176818671 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:19.005835 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7cef49f6-601e-49a2-85a5-7d5d713d1f15 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1a4fd47d-f2c0-407d-9b72-65fe77fadf08,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:19.010542 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-7cef49f6-601e-49a2-85a5-7d5d713d1f15 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:19.019214 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-7cef49f6-601e-49a2-85a5-7d5d713d1f15 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:19.021079 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-7cef49f6-601e-49a2-85a5-7d5d713d1f15 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/1a4fd47d-f2c0-407d-9b72-65fe77fadf08 HTTP/1.1" status: 204 len: 168 time: 0.2008736 May 24 11:27:19.793654 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 33970) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:19.903556 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Request body: {'network': {'name': 'tempest-VolumesSnapshotTestJSON-1277734641-network', 'tenant_id': '094a816f95224ef7b8a6a3b2c60a8640'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:27:19.908405 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:27:19.912522 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Usage tracker for resource:network and tenant:094a816f95224ef7b8a6a3b2c60a8640 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:27:19.915825 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:27:19.923157 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:27:19.933855 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Made reservation on behalf of 094a816f95224ef7b8a6a3b2c60a8640 for: {'network': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:27:19.936454 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:19.948585 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:19.967812 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:27:19.971775 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Usage tracker for resource:security_group_rule and tenant:094a816f95224ef7b8a6a3b2c60a8640 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:27:19.974690 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:27:19.981436 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:27:20.070281 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:20.075326 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:20.078366 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-41bea760-99db-465e-940c-730da2e8db3b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:20.112697 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] vlan segment allocate from pool success with {'vlan_id': 533, 'physical_network': 'physnet1'} {{(pid=91820) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:27:20.115107 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b1a82551-b9a0-47c2-92e9-ff64255c21b2,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:20.118254 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:20.127508 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:20.127972 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.segments_db [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Added segment 239b64d1-3bcb-4407-bb43-8f2cc8e4572c of type vlan for network 107c70c5-f175-4988-a826-f2f05a10d651 May 24 11:27:20.154129 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:20.183121 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:20.184636 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:20.185237 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:20.189712 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:20.288758 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 0.4938681 May 24 11:27:20.291720 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 33976) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:27:20.293736 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Pushing event updated for resources: {'Network': ['ID=107c70c5-f175-4988-a826-f2f05a10d651,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:20.297067 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-ab367c8d-0573-468a-97f7-5325084ff16e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:20.297451 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Request body: {'subnet': {'network_id': '107c70c5-f175-4988-a826-f2f05a10d651', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesSnapshotTestJSON-1277734641-subnet', 'tenant_id': '094a816f95224ef7b8a6a3b2c60a8640', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:27:20.303385 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network 107c70c5-f175-4988-a826-f2f05a10d651 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:27:20.312487 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:27:20.317609 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Usage tracker for resource:subnet and tenant:094a816f95224ef7b8a6a3b2c60a8640 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:27:20.324688 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:27:20.331921 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:27:20.341587 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Made reservation on behalf of 094a816f95224ef7b8a6a3b2c60a8640 for: {'subnet': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:27:20.344132 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-535570794'] for subnet, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:20.729221 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-534944967', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-535570812'] for subnet, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:20.733727 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a7341bf9-d128-490a-bb94-61d655ce9e1b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:20.809185 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Pushing event updated for resources: {'Subnet': ['ID=47be189e-9ef9-43c1-9ea3-7b7c95cb166a,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:20.814568 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:20.815256 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Checking segment: {'id': '239b64d1-3bcb-4407-bb43-8f2cc8e4572c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 533, 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:27:20.815606 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Network hostable DHCP agents. Network: 107c70c5-f175-4988-a826-f2f05a10d651, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91819) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:27:20.832403 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Network 107c70c5-f175-4988-a826-f2f05a10d651 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91819) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:27:20.841638 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, 107c70c5-f175-4988-a826-f2f05a10d651) {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:27:20.850465 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-dae0ff4a-84fa-4223-9cac-d68b58dc890b None None] Network 107c70c5-f175-4988-a826-f2f05a10d651 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:27:20.850842 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:20.851752 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:20.882418 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 851 time: 0.5895839 May 24 11:27:21.056512 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-4204b541-1de5-4054-8ce5-9388d624f586 None None] Network 107c70c5-f175-4988-a826-f2f05a10d651 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:27:22.103178 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:22.917969 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-29f4e817-fde4-4480-86f9-d1c313f75884 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:22.919408 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-29f4e817-fde4-4480-86f9-d1c313f75884 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/networks?id=107c70c5-f175-4988-a826-f2f05a10d651 HTTP/1.1" status: 200 len: 814 time: 0.1477413 May 24 11:27:22.933543 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-30afd195-b37f-45f2-8a02-c8c38a8a29a1 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/quotas/094a816f95224ef7b8a6a3b2c60a8640 HTTP/1.1" status: 200 len: 375 time: 0.0086999 May 24 11:27:22.979897 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-87f6a43f-4d92-46a0-a92e-2e545d1448fe tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=094a816f95224ef7b8a6a3b2c60a8640&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0416639 May 24 11:27:23.038829 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-00497263-b691-4186-8b1e-baad650b3741 service neutron] http://10.222.0.18:9696/v2.0/extensions returned with HTTP 200 {{(pid=91819) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 11:27:23.039803 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-00497263-b691-4186-8b1e-baad650b3741 service neutron] 10.222.0.18 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0536888 May 24 11:27:23.295071 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-912d0e8f-65c4-46d7-9e23-77c0b8e9dc3b service neutron] 10.222.0.18 "GET /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2485411 May 24 11:27:23.429458 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-c6a75519-8b6d-4f2d-8ed6-2602baa39b7c service neutron] 10.222.0.18 "GET /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1283078 May 24 11:27:23.436486 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-ac6b6a09-2627-4439-8a7f-54f1fcf34e42 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:24.684210 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d94bc2dc-facd-443a-8beb-a2bac3497a84 req-9ba13bcd-4b2e-4b8e-8dbd-ee38c1cc9cdc tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 210 time: 0.0442569 May 24 11:27:25.295369 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:27:25.766797 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:25.786235 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d07e2d82-362e-46b8-8c77-ef0764959232 req-fcf6708f-aaee-4aaf-b47b-581bde011f3a tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 210 time: 0.0482435 May 24 11:27:25.886917 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-599b76bd-abe9-41f6-909a-64a274a89ad0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:26.890869 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-80db31d8-0066-4d29-898f-c3e9034bd6bf req-ea5968ed-e688-45e9-8051-fdad47259a15 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 210 time: 0.0534718 May 24 11:27:27.006389 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:27.989543 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-08323fe1-870b-42f6-83b6-91389cfc4590 req-cdc5ec3b-56c8-467c-8e76-62e669398c2e tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 210 time: 0.0447824 May 24 11:27:28.951517 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:29.245926 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b6e98783-6d6d-45c2-b307-60eb1cbe10c3 req-1ffd7a44-32bc-40c6-9b2f-ee9e628bc977 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 210 time: 0.1482944 May 24 11:27:30.060009 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-a69d8224-ebbc-4d4a-91f5-9b96835619fd tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:30.061619 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-a69d8224-ebbc-4d4a-91f5-9b96835619fd tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.93 "GET /v2.0/networks?id=107c70c5-f175-4988-a826-f2f05a10d651 HTTP/1.1" status: 200 len: 814 time: 0.1652801 May 24 11:27:30.074855 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Request body: {'port': {'device_id': '4dfe7c81-604f-4abb-b5cb-9f73d90eb292', 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651', 'admin_state_up': True, 'tenant_id': '094a816f95224ef7b8a6a3b2c60a8640'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:27:30.220528 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:27:30.224406 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Usage tracker for resource:port and tenant:094a816f95224ef7b8a6a3b2c60a8640 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:27:30.227586 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Quota usage for port was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:27:30.233679 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:27:30.244777 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Made reservation on behalf of 094a816f95224ef7b8a6a3b2c60a8640 for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:27:30.248499 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:30.436768 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-e4279899-1009-4a35-bdaa-4a35d71400c8 req-4eec0739-233f-455a-a5cb-59adba4a4146 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 210 time: 0.0535779 May 24 11:27:30.460536 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Allocated IP 172.20.0.3 (107c70c5-f175-4988-a826-f2f05a10d651/47be189e-9ef9-43c1-9ea3-7b7c95cb166a/3dee5892-806d-45af-bf59-cdabb50f251d) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:27:30.669970 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:30.699608 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:30.705133 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:30.772028 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Network 107c70c5-f175-4988-a826-f2f05a10d651 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:27:30.775597 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:27:30.779365 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:27:30.816054 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Pushing event updated for resources: {'Port': ['ID=3dee5892-806d-45af-bf59-cdabb50f251d,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:30.823588 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.118s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:30.854240 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:30.854842 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:30.879586 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-50f5d494-5033-4f18-bf71-6a2f1fddcf0c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 0.8093691 May 24 11:27:30.891381 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Request body: {'port': {'device_id': '4dfe7c81-604f-4abb-b5cb-9f73d90eb292', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:27:31.002590 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:31.451324 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 3dee5892-806d-45af-bf59-cdabb50f251d {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:27:31.458568 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:31.473754 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Transition to ACTIVE for port object 3dee5892-806d-45af-bf59-cdabb50f251d will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:27:31.478778 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:31.482428 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-bd5dcfe1-bb9b-48d8-b3ca-5f784460aa93 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:31.627107 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63dd0423-182c-4392-b835-35f7d5bd4766 req-bbf5e5af-be9d-437e-98af-89f312063406 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:31.628555 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-63dd0423-182c-4392-b835-35f7d5bd4766 req-bbf5e5af-be9d-437e-98af-89f312063406 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 943 time: 0.0816693 May 24 11:27:31.633551 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Pushing event updated for resources: {'Port': ['ID=3dee5892-806d-45af-bf59-cdabb50f251d,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:31.641654 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.159s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:31.675827 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-63dd0423-182c-4392-b835-35f7d5bd4766 req-a7ff4dec-c4fb-4476-92ed-d0fcee8689b8 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Synchronizing usage tracker for tenant:094a816f95224ef7b8a6a3b2c60a8640 on resource:security_group {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:27:31.694658 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-63dd0423-182c-4392-b835-35f7d5bd4766 req-a7ff4dec-c4fb-4476-92ed-d0fcee8689b8 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Unset dirty status for tenant:094a816f95224ef7b8a6a3b2c60a8640 on resource:security_group {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:27:31.695937 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-63dd0423-182c-4392-b835-35f7d5bd4766 req-a7ff4dec-c4fb-4476-92ed-d0fcee8689b8 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0606420 May 24 11:27:31.717735 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Attempting to bind port 3dee5892-806d-45af-bf59-cdabb50f251d on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:27:31.718074 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Attempting to bind port 3dee5892-806d-45af-bf59-cdabb50f251d by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': '239b64d1-3bcb-4407-bb43-8f2cc8e4572c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 533, 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:27:31.718444 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Attempting to bind port 3dee5892-806d-45af-bf59-cdabb50f251d on network 107c70c5-f175-4988-a826-f2f05a10d651 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:27:31.723304 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Port 3dee5892-806d-45af-bf59-cdabb50f251d on network 107c70c5-f175-4988-a826-f2f05a10d651 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:27:31.723654 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Attempting to bind port 3dee5892-806d-45af-bf59-cdabb50f251d on network 107c70c5-f175-4988-a826-f2f05a10d651 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:27:31.728879 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 27, 22), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:27:31.729254 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Checking segment: {'id': '239b64d1-3bcb-4407-bb43-8f2cc8e4572c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 533, 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:27:31.729641 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Bound using segment: {'id': '239b64d1-3bcb-4407-bb43-8f2cc8e4572c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 533, 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:27:31.730073 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Bound port: 3dee5892-806d-45af-bf59-cdabb50f251d, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '239b64d1-3bcb-4407-bb43-8f2cc8e4572c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 533, 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:27:31.730426 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:31.820793 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] For port 3dee5892-806d-45af-bf59-cdabb50f251d, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:27:31.824556 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] For port 3dee5892-806d-45af-bf59-cdabb50f251d, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=3dee5892-806d-45af-bf59-cdabb50f251d,segment=NetworkSegment(239b64d1-3bcb-4407-bb43-8f2cc8e4572c),segment_id=239b64d1-3bcb-4407-bb43-8f2cc8e4572c)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:27:31.831255 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:31.836601 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d3098214-9cbd-419f-8781-4f2c374be97e tempest-AttachVolumeTestJSON-1206692050 tempest-AttachVolumeTestJSON-1206692050-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:31.897037 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:31.897675 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:31.899211 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] 10.222.0.93 "PUT /v2.0/ports/3dee5892-806d-45af-bf59-cdabb50f251d HTTP/1.1" status: 200 len: 1093 time: 1.0119171 May 24 11:27:31.899750 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '4dfe7c81-604f-4abb-b5cb-9f73d90eb292', 'tag': '3dee5892-806d-45af-bf59-cdabb50f251d'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:27:31.923550 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Pushing event updated for resources: {'Port': ['ID=3dee5892-806d-45af-bf59-cdabb50f251d,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:31.927145 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:31.978639 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '4dfe7c81-604f-4abb-b5cb-9f73d90eb292', 'tag': '3dee5892-806d-45af-bf59-cdabb50f251d', 'status': 'completed', 'code': 200} May 24 11:27:32.018989 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-b1e93eee-6f25-4eda-9044-b412b28fbafb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:32.020339 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-b1e93eee-6f25-4eda-9044-b412b28fbafb service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=094a816f95224ef7b8a6a3b2c60a8640&device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 1096 time: 0.0750749 May 24 11:27:32.097018 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-3192991f-4e89-458d-a65c-ff436d2a212c service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=3dee5892-806d-45af-bf59-cdabb50f251d HTTP/1.1" status: 200 len: 217 time: 0.0673358 May 24 11:27:32.161010 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-48d277c5-cadb-462f-bccf-8aebad805e01 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:32.162497 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-48d277c5-cadb-462f-bccf-8aebad805e01 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=47be189e-9ef9-43c1-9ea3-7b7c95cb166a HTTP/1.1" status: 200 len: 854 time: 0.0587428 May 24 11:27:32.213114 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-deb57313-c45b-419a-9817-1b86c1a6fc19 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=107c70c5-f175-4988-a826-f2f05a10d651&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0440149 May 24 11:27:32.348798 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-f9ea202f-7018-420d-b2b5-e0ba9113b7ac service neutron] 10.222.0.93 "GET /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1296980 May 24 11:27:32.484194 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f14ee771-2a99-4e0b-8b68-c0c5a3c9064b req-58f9ca61-5eb7-4e7f-b4e1-e247055f3186 service neutron] 10.222.0.93 "GET /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1286049 May 24 11:27:32.602072 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d6a6874-a0f3-4698-8b55-b205c52bad69 req-e07a7d7b-b8d3-4295-b7f7-e9285b1bf1e7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:32.603500 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d6a6874-a0f3-4698-8b55-b205c52bad69 req-e07a7d7b-b8d3-4295-b7f7-e9285b1bf1e7 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=094a816f95224ef7b8a6a3b2c60a8640&device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 1096 time: 0.0724497 May 24 11:27:32.739045 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d6a6874-a0f3-4698-8b55-b205c52bad69 req-27742450-b002-404b-9c0f-302651b532db service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:32.740496 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d6a6874-a0f3-4698-8b55-b205c52bad69 req-27742450-b002-404b-9c0f-302651b532db service neutron] 10.222.0.93 "GET /v2.0/networks?id=107c70c5-f175-4988-a826-f2f05a10d651 HTTP/1.1" status: 200 len: 915 time: 0.1282151 May 24 11:27:32.814403 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d6a6874-a0f3-4698-8b55-b205c52bad69 req-47d98cc3-5fea-4af1-bb5f-759ed12c1d17 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=3dee5892-806d-45af-bf59-cdabb50f251d HTTP/1.1" status: 200 len: 217 time: 0.0658917 May 24 11:27:32.882614 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d6a6874-a0f3-4698-8b55-b205c52bad69 req-82efa4bb-b327-425f-9d43-0f94c04467c9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:32.884529 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d6a6874-a0f3-4698-8b55-b205c52bad69 req-82efa4bb-b327-425f-9d43-0f94c04467c9 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=47be189e-9ef9-43c1-9ea3-7b7c95cb166a HTTP/1.1" status: 200 len: 854 time: 0.0627954 May 24 11:27:32.888934 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6dc87a53-751d-426d-b266-0bb52f402561 req-14c3d9a7-ddb5-4577-8f0c-7f5da811595b tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:32.890444 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6dc87a53-751d-426d-b266-0bb52f402561 req-14c3d9a7-ddb5-4577-8f0c-7f5da811595b tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 943 time: 0.0868363 May 24 11:27:32.935377 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d6a6874-a0f3-4698-8b55-b205c52bad69 req-32e62afb-e222-4591-a5af-915e3b05720d service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=107c70c5-f175-4988-a826-f2f05a10d651&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0433018 May 24 11:27:32.938574 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6dc87a53-751d-426d-b266-0bb52f402561 req-bf206ad1-797f-4500-bf9a-5a35e00389dc tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0416532 May 24 11:27:33.069461 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d6a6874-a0f3-4698-8b55-b205c52bad69 req-58317f30-cbec-4089-82f5-f7d2520b95d9 service neutron] 10.222.0.93 "GET /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1274147 May 24 11:27:33.293798 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9d6a6874-a0f3-4698-8b55-b205c52bad69 req-88822ca4-6b68-402c-9a26-e116a730b7a1 service neutron] 10.222.0.93 "GET /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2170153 May 24 11:27:34.132735 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4d865ad9-15c7-402f-861c-7f465fbbcc97 req-73e95af5-c3e7-4fa9-9f92-e0f4c41a5cc9 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:34.133538 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-4d865ad9-15c7-402f-861c-7f465fbbcc97 req-73e95af5-c3e7-4fa9-9f92-e0f4c41a5cc9 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Synchronizing usage tracker for tenant:094a816f95224ef7b8a6a3b2c60a8640 on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:27:34.153536 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-4d865ad9-15c7-402f-861c-7f465fbbcc97 req-73e95af5-c3e7-4fa9-9f92-e0f4c41a5cc9 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Unset dirty status for tenant:094a816f95224ef7b8a6a3b2c60a8640 on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:27:34.154729 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4d865ad9-15c7-402f-861c-7f465fbbcc97 req-73e95af5-c3e7-4fa9-9f92-e0f4c41a5cc9 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 943 time: 0.1076696 May 24 11:27:34.203796 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-4d865ad9-15c7-402f-861c-7f465fbbcc97 req-3e83f97c-3284-4aa2-a5e5-dfa1d7302b56 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0425546 May 24 11:27:35.394863 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3233ecb7-e884-47b6-ab60-63502825a1af req-48023df3-ebbc-4691-a4f1-3d5a09a202c5 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:35.396346 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3233ecb7-e884-47b6-ab60-63502825a1af req-48023df3-ebbc-4691-a4f1-3d5a09a202c5 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 943 time: 0.0847025 May 24 11:27:35.443915 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3233ecb7-e884-47b6-ab60-63502825a1af req-9f51894c-4279-4437-b961-fd1cb6e1be27 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0409379 May 24 11:27:36.107394 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 3dee5892-806d-45af-bf59-cdabb50f251d details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:27:36.253607 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 3dee5892-806d-45af-bf59-cdabb50f251d, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=3dee5892-806d-45af-bf59-cdabb50f251d,segment=NetworkSegment(239b64d1-3bcb-4407-bb43-8f2cc8e4572c),segment_id=239b64d1-3bcb-4407-bb43-8f2cc8e4572c)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:27:36.267922 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '3dee5892-806d-45af-bf59-cdabb50f251d', 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651', 'port_id': '3dee5892-806d-45af-bf59-cdabb50f251d', 'mac_address': 'fa:16:3e:ee:b1:23', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 533, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '47be189e-9ef9-43c1-9ea3-7b7c95cb166a', 'ip_address': '172.20.0.3'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:27:36.332856 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:36.349344 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 3dee5892-806d-45af-bf59-cdabb50f251d {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:27:36.365588 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 3dee5892-806d-45af-bf59-cdabb50f251d, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=3dee5892-806d-45af-bf59-cdabb50f251d,segment=NetworkSegment(239b64d1-3bcb-4407-bb43-8f2cc8e4572c),segment_id=239b64d1-3bcb-4407-bb43-8f2cc8e4572c)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:27:36.370711 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:36.372425 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:36.437893 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=3dee5892-806d-45af-bf59-cdabb50f251d,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:36.440943 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:36.575449 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['3dee5892-806d-45af-bf59-cdabb50f251d'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:27:36.643938 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0d07d270-d3f5-48f4-a93b-8407537d0283 req-d6be97d3-8ef2-455d-99e3-ff033452589c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:36.645775 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0d07d270-d3f5-48f4-a93b-8407537d0283 req-d6be97d3-8ef2-455d-99e3-ff033452589c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 944 time: 0.0930107 May 24 11:27:36.693639 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-0d07d270-d3f5-48f4-a93b-8407537d0283 req-830c7d5b-ddbb-456d-94d1-6e71190f3251 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0409577 May 24 11:27:37.878868 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-148f76c4-fa58-479f-ad4c-f47ddf676fb9 req-d43c233b-ebf3-4591-8c39-bc6b60063268 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:37.880325 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-148f76c4-fa58-479f-ad4c-f47ddf676fb9 req-d43c233b-ebf3-4591-8c39-bc6b60063268 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 944 time: 0.0802765 May 24 11:27:37.925691 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-148f76c4-fa58-479f-ad4c-f47ddf676fb9 req-3f4f7917-9a21-404e-a563-bbae6438e104 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0394344 May 24 11:27:38.121782 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 3dee5892-806d-45af-bf59-cdabb50f251d up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:27:38.192982 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 3dee5892-806d-45af-bf59-cdabb50f251d completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:27:38.197836 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 3dee5892-806d-45af-bf59-cdabb50f251d triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:27:38.198639 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:38.322460 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:38.357210 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '4dfe7c81-604f-4abb-b5cb-9f73d90eb292', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '3dee5892-806d-45af-bf59-cdabb50f251d'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:27:38.367884 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 3dee5892-806d-45af-bf59-cdabb50f251d, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=3dee5892-806d-45af-bf59-cdabb50f251d,segment=NetworkSegment(239b64d1-3bcb-4407-bb43-8f2cc8e4572c),segment_id=239b64d1-3bcb-4407-bb43-8f2cc8e4572c)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:27:38.374336 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:38.377279 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-597a084c-17d6-4f49-b1d0-a6d250776b93 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:27:38.450868 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=3dee5892-806d-45af-bf59-cdabb50f251d,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:27:38.454131 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:27:38.687294 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '4dfe7c81-604f-4abb-b5cb-9f73d90eb292', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '3dee5892-806d-45af-bf59-cdabb50f251d', 'code': 200} May 24 11:27:38.920443 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d9feb236-b1bc-4b4f-aee0-e768e9847fe2 req-39fb0cdf-f3b1-4511-b13a-d365c110cd80 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:38.921936 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d9feb236-b1bc-4b4f-aee0-e768e9847fe2 req-39fb0cdf-f3b1-4511-b13a-d365c110cd80 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 1098 time: 0.0766399 May 24 11:27:38.971104 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d9feb236-b1bc-4b4f-aee0-e768e9847fe2 req-0c457ff7-0059-4200-a12b-3c0e00d55e3e service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0429914 May 24 11:27:39.126348 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0c48cd3e-50a8-49f2-9c49-a90faf049bca req-dd14849a-b04c-473c-8db6-98983586ac9d tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:39.127857 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0c48cd3e-50a8-49f2-9c49-a90faf049bca req-dd14849a-b04c-473c-8db6-98983586ac9d tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 945 time: 0.0880001 May 24 11:27:39.176201 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0c48cd3e-50a8-49f2-9c49-a90faf049bca req-7683b0bb-86d2-4710-867d-b21c8d344bfa tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0420187 May 24 11:27:40.366560 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10c66172-3182-4bd9-bd01-4b5583b760f6 req-f64b7a5f-682b-4c1e-ade3-6add6595dd42 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:27:40.368564 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-10c66172-3182-4bd9-bd01-4b5583b760f6 req-f64b7a5f-682b-4c1e-ade3-6add6595dd42 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 945 time: 0.0852482 May 24 11:27:40.416746 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-10c66172-3182-4bd9-bd01-4b5583b760f6 req-cd2de85d-6f34-4bf2-9255-9b45f79d4d08 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0398126 May 24 11:27:52.105865 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:53.439581 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1801eacb-c5e2-4eb5-ac56-d81a22c266f5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:55.776519 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:55.880238 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-beb12203-2b74-46df-9810-d23926640a25 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:57.010324 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:27:58.953776 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:02.294697 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:28:22.104886 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:22.641344 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a06892cd-3835-4489-b23c-e5e1eb89d055 req-5d13928e-3a96-4e0d-85db-03d75c3ee652 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:22.642836 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a06892cd-3835-4489-b23c-e5e1eb89d055 req-5d13928e-3a96-4e0d-85db-03d75c3ee652 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 945 time: 0.0906522 May 24 11:28:22.693836 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-a06892cd-3835-4489-b23c-e5e1eb89d055 req-b282c2e9-80b5-4544-bd75-cd208145255b tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0448329 May 24 11:28:23.444935 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-0539da14-cf51-437e-97ba-67b9a7ed3313 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:23.881908 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e7e57e9-b2e9-4ee2-8d56-fb2908098918 req-3a570578-32da-482b-ba89-68c4ffe9046e tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:23.883451 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2e7e57e9-b2e9-4ee2-8d56-fb2908098918 req-3a570578-32da-482b-ba89-68c4ffe9046e tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 945 time: 0.0822554 May 24 11:28:23.934040 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-2e7e57e9-b2e9-4ee2-8d56-fb2908098918 req-1faf0ab9-500b-4d9b-9bd8-cc7c20b6250c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0448492 May 24 11:28:24.136944 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29afb461-d4b0-4a7f-be15-ac90badce57d req-b9a0d4ba-b4be-41a1-8c41-1ef5656af84b tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:24.138429 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-29afb461-d4b0-4a7f-be15-ac90badce57d req-b9a0d4ba-b4be-41a1-8c41-1ef5656af84b tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.93 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 945 time: 0.0867550 May 24 11:28:24.281072 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-29afb461-d4b0-4a7f-be15-ac90badce57d req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Deleting port 3dee5892-806d-45af-bf59-cdabb50f251d {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:28:24.281483 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-29afb461-d4b0-4a7f-be15-ac90badce57d req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:24.418855 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-29afb461-d4b0-4a7f-be15-ac90badce57d req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] For port 3dee5892-806d-45af-bf59-cdabb50f251d, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=3dee5892-806d-45af-bf59-cdabb50f251d,segment=NetworkSegment(239b64d1-3bcb-4407-bb43-8f2cc8e4572c),segment_id=239b64d1-3bcb-4407-bb43-8f2cc8e4572c)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:28:24.419121 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-29afb461-d4b0-4a7f-be15-ac90badce57d req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:24.543059 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-29afb461-d4b0-4a7f-be15-ac90badce57d req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Calling delete_port for 3dee5892-806d-45af-bf59-cdabb50f251d owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:28:24.677655 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-29afb461-d4b0-4a7f-be15-ac90badce57d req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:24.683892 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:28:24.691785 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['3dee5892-806d-45af-bf59-cdabb50f251d'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:28:24.730919 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Pushing event deleted for resources: {'Port': ['ID=3dee5892-806d-45af-bf59-cdabb50f251d,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:28:24.769941 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:24.770328 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:24.770700 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:24.771065 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:24.771427 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:24.771784 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:24.772139 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:24.772526 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:24.772879 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:24.773260 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:24.773614 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:24.773964 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:24.781247 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.097s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:28:24.821318 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-29afb461-d4b0-4a7f-be15-ac90badce57d req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] No DVR routers for this DVR port 3dee5892-806d-45af-bf59-cdabb50f251d on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:28:24.823774 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-29afb461-d4b0-4a7f-be15-ac90badce57d req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:24.826514 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '4dfe7c81-604f-4abb-b5cb-9f73d90eb292', 'name': 'network-vif-deleted', 'tag': '3dee5892-806d-45af-bf59-cdabb50f251d'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:28:24.843591 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-29afb461-d4b0-4a7f-be15-ac90badce57d req-ef3f2e3d-b8d3-4dc7-9a6a-5960309434d7 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.93 "DELETE /v2.0/ports/3dee5892-806d-45af-bf59-cdabb50f251d HTTP/1.1" status: 204 len: 173 time: 0.6992064 May 24 11:28:24.872708 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '4dfe7c81-604f-4abb-b5cb-9f73d90eb292', 'name': 'network-vif-deleted', 'tag': '3dee5892-806d-45af-bf59-cdabb50f251d', 'status': 'completed', 'code': 200} May 24 11:28:25.103441 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8e9f0ed4-e78f-4f82-b80e-e309bf122515 req-717476dd-2c6c-4772-9d31-f2941c7e5b51 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=4dfe7c81-604f-4abb-b5cb-9f73d90eb292 HTTP/1.1" status: 200 len: 210 time: 0.0429163 May 24 11:28:25.770447 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:25.883690 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c323fd50-d126-48dc-a7a4-1ce275ef9b01 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:27.014968 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:28.521490 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 3dee5892-806d-45af-bf59-cdabb50f251d no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:28:28.558852 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 3dee5892-806d-45af-bf59-cdabb50f251d {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:28:28.559211 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 3dee5892-806d-45af-bf59-cdabb50f251d not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:28:28.958260 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:36.760990 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-e3ce9dc1-03fc-42f4-8678-e477844cb5c0 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:36.762120 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-e3ce9dc1-03fc-42f4-8678-e477844cb5c0 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Synchronizing usage tracker for tenant:094a816f95224ef7b8a6a3b2c60a8640 on resource:network {{(pid=91820) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:28:36.782675 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-e3ce9dc1-03fc-42f4-8678-e477844cb5c0 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Unset dirty status for tenant:094a816f95224ef7b8a6a3b2c60a8640 on resource:network {{(pid=91820) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:28:36.783983 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-e3ce9dc1-03fc-42f4-8678-e477844cb5c0 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/networks?id=107c70c5-f175-4988-a826-f2f05a10d651 HTTP/1.1" status: 200 len: 814 time: 0.2064617 May 24 11:28:36.798987 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-3996c097-3625-4f56-a976-8be534f583b4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/quotas/094a816f95224ef7b8a6a3b2c60a8640 HTTP/1.1" status: 200 len: 375 time: 0.0094082 May 24 11:28:36.851523 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-45c38bd7-31d4-48a8-8861-0a32bc96d7f2 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=094a816f95224ef7b8a6a3b2c60a8640&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0465753 May 24 11:28:36.958213 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-3335ab5e-131f-4e4b-9d97-be5972eb8efe service neutron] http://10.222.0.18:9696/v2.0/extensions returned with HTTP 200 {{(pid=91820) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 24 11:28:36.959212 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-3335ab5e-131f-4e4b-9d97-be5972eb8efe service neutron] 10.222.0.18 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0998394 May 24 11:28:37.097850 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-885c2bc9-5fbd-49f7-a4c0-29550d561843 service neutron] 10.222.0.18 "GET /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1281927 May 24 11:28:37.233978 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-ab3d402c-1a7a-4609-a11a-3f8a44c40632 service neutron] 10.222.0.18 "GET /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1297464 May 24 11:28:37.417401 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-25f612ef-8b44-40cb-afb8-bcf43639572b req-8257f21f-8f08-41c5-9b06-ebd65537704b tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Synchronizing usage tracker for tenant:094a816f95224ef7b8a6a3b2c60a8640 on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:28:37.434019 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-25f612ef-8b44-40cb-afb8-bcf43639572b req-8257f21f-8f08-41c5-9b06-ebd65537704b tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Unset dirty status for tenant:094a816f95224ef7b8a6a3b2c60a8640 on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:28:37.435592 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-25f612ef-8b44-40cb-afb8-bcf43639572b req-8257f21f-8f08-41c5-9b06-ebd65537704b tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 210 time: 0.0605328 May 24 11:28:38.447135 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-12cde9ef-68b4-4e71-be79-84027d4cf467 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:38.448588 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-12cde9ef-68b4-4e71-be79-84027d4cf467 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.93 "GET /v2.0/networks?id=107c70c5-f175-4988-a826-f2f05a10d651 HTTP/1.1" status: 200 len: 814 time: 0.1570241 May 24 11:28:38.460659 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Request body: {'port': {'device_id': '0e8cfb80-57f1-4930-a44e-571cd99d90cc', 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651', 'admin_state_up': True, 'tenant_id': '094a816f95224ef7b8a6a3b2c60a8640'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:28:38.608918 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8a4e1a35-98aa-47f7-9a4c-f447299c033b req-37fc2e9a-07d9-41c5-8156-86f9dc725f7c tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 210 time: 0.0476091 May 24 11:28:38.700967 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:28:38.712417 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:28:38.722222 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Made reservation on behalf of 094a816f95224ef7b8a6a3b2c60a8640 for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:28:38.725693 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:38.893511 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Allocated IP 172.20.0.6 (107c70c5-f175-4988-a826-f2f05a10d651/47be189e-9ef9-43c1-9ea3-7b7c95cb166a/f22209dc-7d4d-4038-8451-e8f3c7cfff2d) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:28:39.099726 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:39.128736 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:39.134880 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:28:39.204505 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Network 107c70c5-f175-4988-a826-f2f05a10d651 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:28:39.207831 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:28:39.211755 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:28:39.247096 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Pushing event updated for resources: {'Port': ['ID=f22209dc-7d4d-4038-8451-e8f3c7cfff2d,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:28:39.251657 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:28:39.279387 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:39.279985 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:39.302766 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:28:39.306553 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c804e151-476b-439a-aa9c-56de1d791862 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 0.8496323 May 24 11:28:39.317752 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Request body: {'port': {'device_id': '0e8cfb80-57f1-4930-a44e-571cd99d90cc', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:28:39.437059 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:39.853164 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id f22209dc-7d4d-4038-8451-e8f3c7cfff2d {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:28:39.862465 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:39.881669 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Transition to ACTIVE for port object f22209dc-7d4d-4038-8451-e8f3c7cfff2d will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:28:39.890957 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-053a1116-61af-424e-8af4-3b2d76264329 req-259b1386-1452-4bd1-ab22-42ac42cbfe7e tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:39.891645 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-053a1116-61af-424e-8af4-3b2d76264329 req-259b1386-1452-4bd1-ab22-42ac42cbfe7e tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Synchronizing usage tracker for tenant:094a816f95224ef7b8a6a3b2c60a8640 on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:28:39.895410 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:39.900908 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-db1c7ee6-215e-400f-835a-c853500ae83d service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:28:40.025734 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-053a1116-61af-424e-8af4-3b2d76264329 req-259b1386-1452-4bd1-ab22-42ac42cbfe7e tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Unset dirty status for tenant:094a816f95224ef7b8a6a3b2c60a8640 on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:28:40.027023 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-053a1116-61af-424e-8af4-3b2d76264329 req-259b1386-1452-4bd1-ab22-42ac42cbfe7e tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 931 time: 0.3007300 May 24 11:28:40.085435 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Pushing event updated for resources: {'Port': ['ID=f22209dc-7d4d-4038-8451-e8f3c7cfff2d,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:28:40.095599 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.195s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:28:40.163726 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-053a1116-61af-424e-8af4-3b2d76264329 req-9d5a6dd4-3d5f-453f-9cd0-56f3a87c34e3 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1286170 May 24 11:28:40.187417 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Attempting to bind port f22209dc-7d4d-4038-8451-e8f3c7cfff2d on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:28:40.187762 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Attempting to bind port f22209dc-7d4d-4038-8451-e8f3c7cfff2d by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': '239b64d1-3bcb-4407-bb43-8f2cc8e4572c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 533, 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:28:40.188096 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Attempting to bind port f22209dc-7d4d-4038-8451-e8f3c7cfff2d on network 107c70c5-f175-4988-a826-f2f05a10d651 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:28:40.192729 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Port f22209dc-7d4d-4038-8451-e8f3c7cfff2d on network 107c70c5-f175-4988-a826-f2f05a10d651 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:28:40.193055 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Attempting to bind port f22209dc-7d4d-4038-8451-e8f3c7cfff2d on network 107c70c5-f175-4988-a826-f2f05a10d651 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:28:40.200199 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 28, 22), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:28:40.200540 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Checking segment: {'id': '239b64d1-3bcb-4407-bb43-8f2cc8e4572c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 533, 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:28:40.200930 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Bound using segment: {'id': '239b64d1-3bcb-4407-bb43-8f2cc8e4572c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 533, 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:28:40.201391 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Bound port: f22209dc-7d4d-4038-8451-e8f3c7cfff2d, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '239b64d1-3bcb-4407-bb43-8f2cc8e4572c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 533, 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:28:40.201745 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:40.290563 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] For port f22209dc-7d4d-4038-8451-e8f3c7cfff2d, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:28:40.294330 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] For port f22209dc-7d4d-4038-8451-e8f3c7cfff2d, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=f22209dc-7d4d-4038-8451-e8f3c7cfff2d,segment=NetworkSegment(239b64d1-3bcb-4407-bb43-8f2cc8e4572c),segment_id=239b64d1-3bcb-4407-bb43-8f2cc8e4572c)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:28:40.298705 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:40.302413 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:28:40.365055 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:40.365718 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:40.367090 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] 10.222.0.93 "PUT /v2.0/ports/f22209dc-7d4d-4038-8451-e8f3c7cfff2d HTTP/1.1" status: 200 len: 1093 time: 1.0535438 May 24 11:28:40.367590 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0e8cfb80-57f1-4930-a44e-571cd99d90cc', 'tag': 'f22209dc-7d4d-4038-8451-e8f3c7cfff2d'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:28:40.388358 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Pushing event updated for resources: {'Port': ['ID=f22209dc-7d4d-4038-8451-e8f3c7cfff2d,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:28:40.391353 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:28:40.447686 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0e8cfb80-57f1-4930-a44e-571cd99d90cc', 'tag': 'f22209dc-7d4d-4038-8451-e8f3c7cfff2d', 'status': 'completed', 'code': 200} May 24 11:28:40.488485 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-820448c4-1c5f-4b51-9061-51dbe98ba752 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:40.490731 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-820448c4-1c5f-4b51-9061-51dbe98ba752 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=094a816f95224ef7b8a6a3b2c60a8640&device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 1096 time: 0.0763788 May 24 11:28:40.663332 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-2979e8e9-9909-42f8-b65e-67ab0dfd4f8f service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=f22209dc-7d4d-4038-8451-e8f3c7cfff2d HTTP/1.1" status: 200 len: 217 time: 0.1668558 May 24 11:28:40.728130 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-42d5c65a-e456-4958-8abb-ca66ec3243ad service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:40.729583 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-42d5c65a-e456-4958-8abb-ca66ec3243ad service neutron] 10.222.0.93 "GET /v2.0/subnets?id=47be189e-9ef9-43c1-9ea3-7b7c95cb166a HTTP/1.1" status: 200 len: 854 time: 0.0607555 May 24 11:28:40.777401 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-ad3af595-dbaf-4ea9-a9d7-2e203fed9be8 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=107c70c5-f175-4988-a826-f2f05a10d651&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0419965 May 24 11:28:40.920004 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-4b36e915-0cb1-42fb-9fb6-22101a306bae service neutron] 10.222.0.93 "GET /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1365631 May 24 11:28:41.065529 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-cd41101d-ba43-487e-b0fe-3661e1d62c7f req-e240552c-e11b-4c9e-b242-720875c9c8b8 service neutron] 10.222.0.93 "GET /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1395729 May 24 11:28:41.190001 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f6bcd324-52b0-44bf-a22b-8fe33b5e28d4 req-3461a303-4871-4be9-9f62-ccf6d5971362 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:41.191601 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f6bcd324-52b0-44bf-a22b-8fe33b5e28d4 req-3461a303-4871-4be9-9f62-ccf6d5971362 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=094a816f95224ef7b8a6a3b2c60a8640&device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 1096 time: 0.0774455 May 24 11:28:41.384261 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f6bcd324-52b0-44bf-a22b-8fe33b5e28d4 req-81f96b2a-b21a-4a8d-89c3-515c19485e41 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:41.386119 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f6bcd324-52b0-44bf-a22b-8fe33b5e28d4 req-81f96b2a-b21a-4a8d-89c3-515c19485e41 service neutron] 10.222.0.93 "GET /v2.0/networks?id=107c70c5-f175-4988-a826-f2f05a10d651 HTTP/1.1" status: 200 len: 915 time: 0.1879103 May 24 11:28:41.469308 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f6bcd324-52b0-44bf-a22b-8fe33b5e28d4 req-ed61836f-9bdd-42bd-913a-75e18dc9264a service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=f22209dc-7d4d-4038-8451-e8f3c7cfff2d HTTP/1.1" status: 200 len: 217 time: 0.0711539 May 24 11:28:41.516655 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3e5f7934-bda8-4a89-947c-8fc282e3eb31 req-2052744f-3f85-49b7-adae-ef6957160326 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:41.518152 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3e5f7934-bda8-4a89-947c-8fc282e3eb31 req-2052744f-3f85-49b7-adae-ef6957160326 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 943 time: 0.2287149 May 24 11:28:41.571950 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f6bcd324-52b0-44bf-a22b-8fe33b5e28d4 req-9a12e5d6-8346-48b6-a63c-8225d38b1208 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:41.573414 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f6bcd324-52b0-44bf-a22b-8fe33b5e28d4 req-9a12e5d6-8346-48b6-a63c-8225d38b1208 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=47be189e-9ef9-43c1-9ea3-7b7c95cb166a HTTP/1.1" status: 200 len: 854 time: 0.0969746 May 24 11:28:41.619834 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3e5f7934-bda8-4a89-947c-8fc282e3eb31 req-1577445e-c34a-49bf-bc77-55c885d30426 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0927269 May 24 11:28:41.626613 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f6bcd324-52b0-44bf-a22b-8fe33b5e28d4 req-bb920a6e-7abf-4c1e-ad4b-480cc222e090 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=107c70c5-f175-4988-a826-f2f05a10d651&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0452509 May 24 11:28:41.770373 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f6bcd324-52b0-44bf-a22b-8fe33b5e28d4 req-4e69a609-1083-4322-a983-1e14447b18f7 service neutron] 10.222.0.93 "GET /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1353765 May 24 11:28:41.910270 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f6bcd324-52b0-44bf-a22b-8fe33b5e28d4 req-770f33cc-7324-42ec-9eb4-f448ec2aa3b2 service neutron] 10.222.0.93 "GET /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1326933 May 24 11:28:42.819072 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05db7960-c7ef-45b0-81a6-429eb0bbe6b5 req-5850db8f-769a-4938-b6bb-9dd22c19fc89 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:42.820423 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-05db7960-c7ef-45b0-81a6-429eb0bbe6b5 req-5850db8f-769a-4938-b6bb-9dd22c19fc89 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 943 time: 0.0864627 May 24 11:28:42.868969 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-05db7960-c7ef-45b0-81a6-429eb0bbe6b5 req-754e894e-075d-4312-a5d9-b3e694cb19b5 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0426393 May 24 11:28:44.063312 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b493b08e-c32d-4649-92c0-b473a55d6eca req-d544b598-a0ae-43a1-a7e7-93784774b692 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:44.064751 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b493b08e-c32d-4649-92c0-b473a55d6eca req-d544b598-a0ae-43a1-a7e7-93784774b692 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 943 time: 0.0876048 May 24 11:28:44.109113 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b493b08e-c32d-4649-92c0-b473a55d6eca req-cf68310d-75be-45a2-a911-2c69ff381241 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0386057 May 24 11:28:44.416004 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device f22209dc-7d4d-4038-8451-e8f3c7cfff2d details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:28:44.552741 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port f22209dc-7d4d-4038-8451-e8f3c7cfff2d, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=f22209dc-7d4d-4038-8451-e8f3c7cfff2d,segment=NetworkSegment(239b64d1-3bcb-4407-bb43-8f2cc8e4572c),segment_id=239b64d1-3bcb-4407-bb43-8f2cc8e4572c)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:28:44.567142 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': 'f22209dc-7d4d-4038-8451-e8f3c7cfff2d', 'network_id': '107c70c5-f175-4988-a826-f2f05a10d651', 'port_id': 'f22209dc-7d4d-4038-8451-e8f3c7cfff2d', 'mac_address': 'fa:16:3e:6a:12:aa', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 533, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '47be189e-9ef9-43c1-9ea3-7b7c95cb166a', 'ip_address': '172.20.0.6'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:28:44.624366 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:44.641942 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id f22209dc-7d4d-4038-8451-e8f3c7cfff2d {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:28:44.658868 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port f22209dc-7d4d-4038-8451-e8f3c7cfff2d, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=f22209dc-7d4d-4038-8451-e8f3c7cfff2d,segment=NetworkSegment(239b64d1-3bcb-4407-bb43-8f2cc8e4572c),segment_id=239b64d1-3bcb-4407-bb43-8f2cc8e4572c)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:28:44.663515 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:44.665725 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:28:44.766976 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=f22209dc-7d4d-4038-8451-e8f3c7cfff2d,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:28:44.775143 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:44.775489 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:44.775830 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:44.776165 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:44.776548 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:44.776881 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:44.777245 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:44.777568 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:44.777896 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:44.778239 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:44.778577 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:44.779137 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:28:44.784420 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.119s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:28:44.865409 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['f22209dc-7d4d-4038-8451-e8f3c7cfff2d'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:28:45.297666 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6506cf80-b3b1-4b47-98b8-8ce0bbc3af1e req-35330155-1d80-44d4-80ce-8c1bbf286570 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:45.299164 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6506cf80-b3b1-4b47-98b8-8ce0bbc3af1e req-35330155-1d80-44d4-80ce-8c1bbf286570 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 944 time: 0.0832727 May 24 11:28:45.345173 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-6506cf80-b3b1-4b47-98b8-8ce0bbc3af1e req-f61cee9a-4cec-4158-95f2-89d087038294 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0396435 May 24 11:28:46.431306 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device f22209dc-7d4d-4038-8451-e8f3c7cfff2d up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:28:46.502725 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port f22209dc-7d4d-4038-8451-e8f3c7cfff2d completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:28:46.508817 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port f22209dc-7d4d-4038-8451-e8f3c7cfff2d triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:28:46.509242 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:46.562352 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2f831b1-15a3-4cbc-af38-4e349bb63f54 req-5847c9bb-7689-464a-831a-54b0273f19b6 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:46.562352 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d2f831b1-15a3-4cbc-af38-4e349bb63f54 req-5847c9bb-7689-464a-831a-54b0273f19b6 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 944 time: 0.1072350 May 24 11:28:46.607892 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d2f831b1-15a3-4cbc-af38-4e349bb63f54 req-7fac2fea-3799-4787-a43c-9d857e2fb540 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0412745 May 24 11:28:46.651855 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:46.679824 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0e8cfb80-57f1-4930-a44e-571cd99d90cc', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'f22209dc-7d4d-4038-8451-e8f3c7cfff2d'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:28:46.689736 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port f22209dc-7d4d-4038-8451-e8f3c7cfff2d, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=f22209dc-7d4d-4038-8451-e8f3c7cfff2d,segment=NetworkSegment(239b64d1-3bcb-4407-bb43-8f2cc8e4572c),segment_id=239b64d1-3bcb-4407-bb43-8f2cc8e4572c)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:28:46.694891 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:46.696730 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:28:46.731906 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0e8cfb80-57f1-4930-a44e-571cd99d90cc', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'f22209dc-7d4d-4038-8451-e8f3c7cfff2d', 'code': 200} May 24 11:28:46.764096 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=f22209dc-7d4d-4038-8451-e8f3c7cfff2d,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:28:46.767143 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:28:46.980645 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85fd10d8-39d8-476a-aebb-b626be08f8eb req-600c61f6-c220-486d-892a-31fa9b3a2880 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:46.989652 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-85fd10d8-39d8-476a-aebb-b626be08f8eb req-600c61f6-c220-486d-892a-31fa9b3a2880 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 1098 time: 0.0776248 May 24 11:28:47.025403 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-85fd10d8-39d8-476a-aebb-b626be08f8eb req-6994d878-6fa4-4280-989b-bf205c340413 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0369012 May 24 11:28:47.831223 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dabbcc98-3824-4b96-a18a-afee788d70f4 req-faac1637-d3ca-4c8f-bdbe-7ca5152c6982 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:47.833718 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dabbcc98-3824-4b96-a18a-afee788d70f4 req-faac1637-d3ca-4c8f-bdbe-7ca5152c6982 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 945 time: 0.1055396 May 24 11:28:47.879688 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dabbcc98-3824-4b96-a18a-afee788d70f4 req-3bd0cf7e-09ba-46d8-a80a-38ee31e363e8 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0404434 May 24 11:28:49.073679 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-af83bf35-3510-4377-a415-e0494fd1ee46 req-ec849376-c32f-4ec5-8727-4eeeda8a42a1 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:28:49.075197 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-af83bf35-3510-4377-a415-e0494fd1ee46 req-ec849376-c32f-4ec5-8727-4eeeda8a42a1 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 945 time: 0.0845916 May 24 11:28:49.122048 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-af83bf35-3510-4377-a415-e0494fd1ee46 req-3af9479b-eae5-439b-8b0d-4bf179d06564 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0410299 May 24 11:28:52.102603 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:53.452614 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aed44717-b256-4384-aeac-704758421182 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:55.774701 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:55.887194 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c3e824f6-6746-4fab-9ee6-e18ca2c1b2a2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:57.019062 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:28:58.965179 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:11.810885 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7a842976-0284-4479-8740-0b4377c532a8 req-74d69b14-4962-49da-b3a3-ba562835fc07 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:29:11.812352 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7a842976-0284-4479-8740-0b4377c532a8 req-74d69b14-4962-49da-b3a3-ba562835fc07 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 945 time: 0.0972009 May 24 11:29:11.859901 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7a842976-0284-4479-8740-0b4377c532a8 req-e65dbe66-6268-42a5-a8f6-aa43db865f60 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0416617 May 24 11:29:13.160994 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07965206-2b75-4cf3-a5c3-311096b8cfdb req-c3ef45a4-8e31-4e31-b71d-55665f2d50a9 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:29:13.162421 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-07965206-2b75-4cf3-a5c3-311096b8cfdb req-c3ef45a4-8e31-4e31-b71d-55665f2d50a9 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 945 time: 0.1851943 May 24 11:29:13.249760 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-07965206-2b75-4cf3-a5c3-311096b8cfdb req-71325f09-c719-42b9-9280-0dadc140b529 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/security-groups?id=b1a82551-b9a0-47c2-92e9-ff64255c21b2&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0817139 May 24 11:29:13.286912 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1c26f31b-6145-477d-b5eb-c541c6b8a9eb req-29d303c4-1072-4519-a5ae-332d60d53a1e tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:29:13.289371 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1c26f31b-6145-477d-b5eb-c541c6b8a9eb req-29d303c4-1072-4519-a5ae-332d60d53a1e tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.93 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 945 time: 0.0921011 May 24 11:29:13.430549 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-1c26f31b-6145-477d-b5eb-c541c6b8a9eb req-6f403013-1b17-40dc-b934-2883102cded4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Deleting port f22209dc-7d4d-4038-8451-e8f3c7cfff2d {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:29:13.430803 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1c26f31b-6145-477d-b5eb-c541c6b8a9eb req-6f403013-1b17-40dc-b934-2883102cded4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:13.570398 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-1c26f31b-6145-477d-b5eb-c541c6b8a9eb req-6f403013-1b17-40dc-b934-2883102cded4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] For port f22209dc-7d4d-4038-8451-e8f3c7cfff2d, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=f22209dc-7d4d-4038-8451-e8f3c7cfff2d,segment=NetworkSegment(239b64d1-3bcb-4407-bb43-8f2cc8e4572c),segment_id=239b64d1-3bcb-4407-bb43-8f2cc8e4572c)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:29:13.570756 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1c26f31b-6145-477d-b5eb-c541c6b8a9eb req-6f403013-1b17-40dc-b934-2883102cded4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:13.698464 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-1c26f31b-6145-477d-b5eb-c541c6b8a9eb req-6f403013-1b17-40dc-b934-2883102cded4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Calling delete_port for f22209dc-7d4d-4038-8451-e8f3c7cfff2d owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:29:13.823903 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1c26f31b-6145-477d-b5eb-c541c6b8a9eb req-6f403013-1b17-40dc-b934-2883102cded4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:13.829745 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e28d571c-aeb2-49fa-9cfd-a6713096cbbb tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:13.839565 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['f22209dc-7d4d-4038-8451-e8f3c7cfff2d'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:29:13.887160 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6f403013-1b17-40dc-b934-2883102cded4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Pushing event deleted for resources: {'Port': ['ID=f22209dc-7d4d-4038-8451-e8f3c7cfff2d,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:13.890301 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-6f403013-1b17-40dc-b934-2883102cded4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.061s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:13.961740 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-1c26f31b-6145-477d-b5eb-c541c6b8a9eb req-6f403013-1b17-40dc-b934-2883102cded4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] No DVR routers for this DVR port f22209dc-7d4d-4038-8451-e8f3c7cfff2d on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:29:13.964163 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-1c26f31b-6145-477d-b5eb-c541c6b8a9eb req-6f403013-1b17-40dc-b934-2883102cded4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:13.966832 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0e8cfb80-57f1-4930-a44e-571cd99d90cc', 'name': 'network-vif-deleted', 'tag': 'f22209dc-7d4d-4038-8451-e8f3c7cfff2d'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:29:13.982110 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1c26f31b-6145-477d-b5eb-c541c6b8a9eb req-6f403013-1b17-40dc-b934-2883102cded4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.93 "DELETE /v2.0/ports/f22209dc-7d4d-4038-8451-e8f3c7cfff2d HTTP/1.1" status: 204 len: 173 time: 0.6883726 May 24 11:29:14.010644 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0e8cfb80-57f1-4930-a44e-571cd99d90cc', 'name': 'network-vif-deleted', 'tag': 'f22209dc-7d4d-4038-8451-e8f3c7cfff2d', 'status': 'completed', 'code': 200} May 24 11:29:14.415310 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-9ce1550e-4f34-441e-802d-ecc9050d132b req-d2eb15d2-75d5-426d-ad2c-1d7140b23161 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Synchronizing usage tracker for tenant:094a816f95224ef7b8a6a3b2c60a8640 on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:29:14.432020 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-9ce1550e-4f34-441e-802d-ecc9050d132b req-d2eb15d2-75d5-426d-ad2c-1d7140b23161 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Unset dirty status for tenant:094a816f95224ef7b8a6a3b2c60a8640 on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:29:14.433416 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9ce1550e-4f34-441e-802d-ecc9050d132b req-d2eb15d2-75d5-426d-ad2c-1d7140b23161 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] 10.222.0.18 "GET /v2.0/ports?device_id=0e8cfb80-57f1-4930-a44e-571cd99d90cc HTTP/1.1" status: 200 len: 210 time: 0.0624185 May 24 11:29:16.293773 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:29:16.632385 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device f22209dc-7d4d-4038-8451-e8f3c7cfff2d no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:29:16.667168 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: f22209dc-7d4d-4038-8451-e8f3c7cfff2d {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:29:16.667518 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device f22209dc-7d4d-4038-8451-e8f3c7cfff2d not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:29:22.099709 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:23.451441 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-27d8fb31-2f22-4566-82b6-1704cc2633b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:25.778269 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:25.889994 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-6f402800-1eb7-423d-a4c3-453f264090ff None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:27.020822 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:28.965300 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:41.073437 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34578) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:41.133845 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_v2 [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Deleting subnet 47be189e-9ef9-43c1-9ea3-7b7c95cb166a {{(pid=91819) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:29:41.137681 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:41.197974 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534853172'] for subnet, precommit_delete_associations {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:41.253313 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:41.503036 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534944900', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535570805'] for subnet, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:41.506913 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-90653e2e-c351-420a-936b-de9f0f347f4c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:41.541483 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Pushing event deleted for resources: {'Subnet': ['ID=47be189e-9ef9-43c1-9ea3-7b7c95cb166a,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:41.546602 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:41.554509 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:41.554917 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:41.555289 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:41.555659 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:41.556015 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:41.556516 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:41.556747 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:41.557114 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:41.557456 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:41.557808 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:41.558156 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:41.558505 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:41.562120 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:41.563915 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] 10.222.0.18 "DELETE /v2.0/subnets/47be189e-9ef9-43c1-9ea3-7b7c95cb166a HTTP/1.1" status: 204 len: 168 time: 0.4894624 May 24 11:29:41.567018 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34582) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:41.678426 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:41.910245 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:41.923567 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['107c70c5-f175-4988-a826-f2f05a10d651']}} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:41.936556 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '239b64d1-3bcb-4407-bb43-8f2cc8e4572c') {'for_net_delete': True} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:41.936908 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '239b64d1-3bcb-4407-bb43-8f2cc8e4572c') {} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:41.948547 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534853183'] for segment, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:41.965579 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:41.973114 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] Releasing vlan 533 on physical network physnet1 to pool {{(pid=91819) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:29:41.973592 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535155490'] for segment, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: Traceback (most recent call last): May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: timer() May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: cb(*args, **kw) May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: return func(*args, **kwargs) May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: self._notify() May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: self.callback(batched_events) May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: event.method(event) May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: return f(self, *a, **k) May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._get(url).json() May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: return self.request(url, 'GET', **kwargs) May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:29:41.986837 dv-cinder-784092-9-iscsi neutron-server[91819]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:29:42.066888 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:42.072483 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:42.074932 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-f5be7b07-8a26-4cd1-b0a8-65fd9663408b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:42.098789 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] Pushing event deleted for resources: {'Network': ['ID=107c70c5-f175-4988-a826-f2f05a10d651,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:42.101252 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:42.108119 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] 10.222.0.18 "DELETE /v2.0/networks/107c70c5-f175-4988-a826-f2f05a10d651 HTTP/1.1" status: 204 len: 168 time: 0.5399344 May 24 11:29:42.273663 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34588) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:42.327636 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-23be0cf7-dec9-485b-80bb-cce586f6ef3b admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=094a816f95224ef7b8a6a3b2c60a8640&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0525908 May 24 11:29:42.330143 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34592) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:42.401580 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-608b4cb6-e15e-48f7-8996-e08baf7e2f1e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:42.429375 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-608b4cb6-e15e-48f7-8996-e08baf7e2f1e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:42.484940 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-608b4cb6-e15e-48f7-8996-e08baf7e2f1e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:42.489023 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-6bcd0690-0d52-43ed-97f9-f2c0f7dc1885 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:42.501455 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-608b4cb6-e15e-48f7-8996-e08baf7e2f1e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b1a82551-b9a0-47c2-92e9-ff64255c21b2,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:42.505198 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-608b4cb6-e15e-48f7-8996-e08baf7e2f1e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:42.515411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-608b4cb6-e15e-48f7-8996-e08baf7e2f1e admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:42.516659 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-608b4cb6-e15e-48f7-8996-e08baf7e2f1e admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/b1a82551-b9a0-47c2-92e9-ff64255c21b2 HTTP/1.1" status: 204 len: 168 time: 0.1849854 May 24 11:29:43.359026 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34616) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:43.477813 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Request body: {'network': {'name': 'tempest-TestShelveInstance-478081922-network', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:43.482612 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:43.486434 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Usage tracker for resource:network and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:43.489768 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:43.496176 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:43.506997 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Made reservation on behalf of 4ee563f50afe4088aebf60ed28c59ea7 for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:29:43.509355 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:43.521763 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:43.536094 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:43.539926 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Usage tracker for resource:security_group_rule and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:43.542704 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:43.548451 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:43.636538 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:43.640703 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:43.643784 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ec483f4a-d564-4419-a290-eb1922d827ba admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:43.675092 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] vlan segment allocate from pool success with {'vlan_id': 540, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:29:43.680958 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c8a3db81-e029-4b49-99b9-010fa99fdcdd,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:43.684741 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:43.688351 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:43.689259 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Added segment 417bf2a8-a5cf-497b-a631-e0f581048962 of type vlan for network 496a7264-ce0a-4dd1-b29d-67154eb68a39 May 24 11:29:43.720088 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:43.750541 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:43.752152 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:29:43.753317 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:43.756628 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:43.861191 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Pushing event updated for resources: {'Network': ['ID=496a7264-ce0a-4dd1-b29d-67154eb68a39,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:43.867190 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 0.5041306 May 24 11:29:43.867190 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-246e92d7-ae14-4b9b-9648-102184070973 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:43.868186 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 34622) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:43.871707 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network 496a7264-ce0a-4dd1-b29d-67154eb68a39 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:29:43.875563 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Request body: {'subnet': {'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestShelveInstance-478081922-subnet', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'ip_version': 4}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:43.880502 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:43.884775 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Usage tracker for resource:subnet and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:43.889452 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:43.896307 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:43.907028 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Made reservation on behalf of 4ee563f50afe4088aebf60ed28c59ea7 for: {'subnet': 1} {{(pid=91820) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:29:43.910252 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-535456753'] for subnet, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:44.397068 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-534788555', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-535456743'] for subnet, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:44.401360 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-3bc706f6-38b1-4e63-9699-f135ea5d1341 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:44.471176 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Pushing event updated for resources: {'Subnet': ['ID=86fc06b2-ed33-40b0-849c-081575fab447,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:44.483774 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:44.484146 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:44.484511 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:44.484938 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:44.485257 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:44.485620 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:44.485978 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:44.486321 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:44.487187 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:44.487535 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:44.487879 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:44.488222 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:44.492966 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:44.493844 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Checking segment: {'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91820) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:29:44.494191 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Network hostable DHCP agents. Network: 496a7264-ce0a-4dd1-b29d-67154eb68a39, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91820) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:29:44.513082 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Network 496a7264-ce0a-4dd1-b29d-67154eb68a39 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91820) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:29:44.523661 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, 496a7264-ce0a-4dd1-b29d-67154eb68a39) {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:29:44.533348 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:29:44.534459 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:44.537439 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e91178f1-4701-4ac1-8682-e13db72bfbd5 None None] Network 496a7264-ce0a-4dd1-b29d-67154eb68a39 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:29:44.560337 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-b01662a5-703a-466d-83c4-6b6686e013c5 admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 844 time: 0.6898203 May 24 11:29:44.560887 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 34626) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:44.566152 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Request body: {'router': {'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:44.571116 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:44.575089 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Usage tracker for resource:router and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:44.578635 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:44.585051 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:44.596763 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-684314', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4157'] for router, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:44.598088 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-684570', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-687045', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-715560', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4164'] for router, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:44.613811 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:44.765423 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpea482e27-f15a-5756-a111-7e0dba035002-496a7264-ce0a-4dd1-b29d-67154eb68a39', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'fixed_ips': [{'subnet_id': '86fc06b2-ed33-40b0-849c-081575fab447'}]}} from dv-cinder-784092-9-iscsi. {{(pid=91821) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 11:29:44.766267 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:44.889405 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-687072', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-684328', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4178'] for router, precommit_update {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:44.940703 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.db_base_plugin_common [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Allocated IP 172.20.0.2 (496a7264-ce0a-4dd1-b29d-67154eb68a39/86fc06b2-ed33-40b0-849c-081575fab447/0c7b8795-f4a5-424e-8b50-5a5d254488ad) {{(pid=91821) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:29:45.156677 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Notify callbacks [] for port, precommit_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:45.179122 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Transition to ACTIVE for port object 0c7b8795-f4a5-424e-8b50-5a5d254488ad will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:29:45.204699 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Transition to ACTIVE for port object 0c7b8795-f4a5-424e-8b50-5a5d254488ad will not be triggered until provisioned by entity DHCP. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:29:45.213311 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:45.217919 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:45.222734 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:45.252670 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:45.287451 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Network 496a7264-ce0a-4dd1-b29d-67154eb68a39 is already hosted by enough agents. {{(pid=91821) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:29:45.290555 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.base_scheduler [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Resources bound (agent type, host, resource id): {{(pid=91821) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:29:45.294407 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Received port after_create {{(pid=91821) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:29:45.325437 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Pushing event updated for resources: {'Port': ['ID=0c7b8795-f4a5-424e-8b50-5a5d254488ad,revision_number=1']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:45.341747 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:45.342032 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:45.342329 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:45.342664 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:45.342993 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:45.343333 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:45.343631 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:45.344011 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:45.344268 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:45.344584 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:45.344899 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:45.345242 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:45.379537 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.162s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:45.399573 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.db_base_plugin_common [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Allocated IP 172.27.0.185 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/0b304a09-229c-496f-b22c-42da5ad3c052) {{(pid=91820) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:29:45.401374 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Attempting to bind port 0c7b8795-f4a5-424e-8b50-5a5d254488ad on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:29:45.401970 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Attempting to bind port 0c7b8795-f4a5-424e-8b50-5a5d254488ad by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:29:45.402379 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Attempting to bind port 0c7b8795-f4a5-424e-8b50-5a5d254488ad on network 496a7264-ce0a-4dd1-b29d-67154eb68a39 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:29:45.409131 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 29, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:29:45.409424 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Checking segment: {'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:29:45.409839 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:45.410238 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Bound using segment: {'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:29:45.410756 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Bound port: 0c7b8795-f4a5-424e-8b50-5a5d254488ad, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:29:45.411137 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:45.512414 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] For port 0c7b8795-f4a5-424e-8b50-5a5d254488ad, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:29:45.516265 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] For port 0c7b8795-f4a5-424e-8b50-5a5d254488ad, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=0c7b8795-f4a5-424e-8b50-5a5d254488ad,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:29:45.523523 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:45.527638 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:45.561391 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:45.600754 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535262843'] for port, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:45.604982 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-5ab4ca59-0e26-4d9d-ba1b-bf6a545c77a9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:45.664642 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Pushing event updated for resources: {'Port': ['ID=0c7b8795-f4a5-424e-8b50-5a5d254488ad,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:45.667521 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91820) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:29:45.670726 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.base_scheduler [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Resources bound (agent type, host, resource id): {{(pid=91820) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:29:45.672992 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c9ff145d-6201-4d23-869d-417dc6f8221b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.145s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:45.675126 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Received port after_create {{(pid=91820) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:29:45.827642 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Pushing event updated for resources: {'Port': ['ID=0b304a09-229c-496f-b22c-42da5ad3c052,revision_number=1']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:45.834159 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.229s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:46.240995 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:46.608171 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-684307', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-687011'] for router, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:46.628436 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-be491d02-93e8-4cc6-9c38-464393f271ee None None] Network 496a7264-ce0a-4dd1-b29d-67154eb68a39 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:29:46.721431 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-11785d5d-59d9-4fe8-b54f-41fb3d01b48b None None] Provisioning for port 0c7b8795-f4a5-424e-8b50-5a5d254488ad completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:29:46.848979 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 is scheduled to L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 with binding_index 1 {{(pid=91820) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 11:29:46.864844 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 0c7b8795-f4a5-424e-8b50-5a5d254488ad up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:29:46.975198 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 0c7b8795-f4a5-424e-8b50-5a5d254488ad completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:29:46.981809 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 0c7b8795-f4a5-424e-8b50-5a5d254488ad triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:46.982444 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:47.045909 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91820) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:29:47.078592 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] Notify callbacks [] for router, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:47.082337 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Sync routers for ids ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:29:47.082692 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34630) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:47.083175 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-2edbe4ad-e924-4e2a-bce5-1a270c026494 admin admin] 10.222.0.18 "POST /v2.0/routers HTTP/1.1" status: 201 len: 920 time: 2.5209451 May 24 11:29:47.124384 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Routers {'2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:29:47.125960 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:29:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:47.240328 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:47.320180 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 0c7b8795-f4a5-424e-8b50-5a5d254488ad, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=0c7b8795-f4a5-424e-8b50-5a5d254488ad,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:29:47.329181 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-a6ddb9a3-25b5-4328-9c65-daec148e65fe None None] Provisioning complete for port 0c7b8795-f4a5-424e-8b50-5a5d254488ad triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:47.329561 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a6ddb9a3-25b5-4328-9c65-daec148e65fe None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:47.362908 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:47.372659 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-079942fb-d8ed-44df-8cf6-989bee881074 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:47.445931 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Set router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:29:47.646970 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=0c7b8795-f4a5-424e-8b50-5a5d254488ad,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:47.650819 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.278s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:47.819310 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:47.823725 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:29:46Z', 'revision_number': 3, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:45Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:47.827764 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Checking router: 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:29:47.828140 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Updating router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 port 0b304a09-229c-496f-b22c-42da5ad3c052 binding host dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:29:47.892448 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:48.165018 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Allocated IP 172.20.0.1 (496a7264-ce0a-4dd1-b29d-67154eb68a39/86fc06b2-ed33-40b0-849c-081575fab447/a61e4d23-b4eb-4110-a731-0d50771b3303) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:29:48.298413 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Notify callbacks [] for port, precommit_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:48.311082 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Transition to ACTIVE for port object 0b304a09-229c-496f-b22c-42da5ad3c052 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:29:48.319120 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:48.323002 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:48.339947 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:48.355922 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Transition to ACTIVE for port object a61e4d23-b4eb-4110-a731-0d50771b3303 will not be triggered until provisioned by entity DHCP. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:29:48.382195 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:48.386120 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-4c6b597b-47de-4ccc-9442-a8d4b6fb45a6 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:48.403454 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Attempting to bind port 0b304a09-229c-496f-b22c-42da5ad3c052 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:29:48.403820 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Attempting to bind port 0b304a09-229c-496f-b22c-42da5ad3c052 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:29:48.404194 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Attempting to bind port 0b304a09-229c-496f-b22c-42da5ad3c052 on network 858b8fad-a73f-4578-9079-6e731b9ff7e2 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:29:48.406818 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3049afe5-bb33-40f9-8689-6286521d9591 None None] Pushing event updated for resources: {'Port': ['ID=0b304a09-229c-496f-b22c-42da5ad3c052,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:48.410579 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-3049afe5-bb33-40f9-8689-6286521d9591 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:48.413199 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 29, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:29:48.413717 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Checking segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:29:48.413996 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:48.414390 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Bound using segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:29:48.415164 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Bound port: 0b304a09-229c-496f-b22c-42da5ad3c052, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:29:48.415292 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:48.464817 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Network 496a7264-ce0a-4dd1-b29d-67154eb68a39 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:29:48.467740 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:29:48.471813 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:29:48.494834 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] For port 0b304a09-229c-496f-b22c-42da5ad3c052, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:29:48.498723 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] For port 0b304a09-229c-496f-b22c-42da5ad3c052, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=0b304a09-229c-496f-b22c-42da5ad3c052,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:29:48.504843 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:48.506702 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Pushing event updated for resources: {'Port': ['ID=a61e4d23-b4eb-4110-a731-0d50771b3303,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:48.511339 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.125s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:48.516009 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:48.531833 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-687570'] for router_interface, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:48.607950 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Routers {'2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:29:48.608591 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:29:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:48.664138 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3049afe5-bb33-40f9-8689-6286521d9591 None None] Pushing event updated for resources: {'Port': ['ID=0b304a09-229c-496f-b22c-42da5ad3c052,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:48.722046 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-3049afe5-bb33-40f9-8689-6286521d9591 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.206s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:48.789165 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-044e04b3-87fd-4d5e-aeeb-5aff98b64222 None None] Provisioning for port a61e4d23-b4eb-4110-a731-0d50771b3303 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:29:48.797368 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-044e04b3-87fd-4d5e-aeeb-5aff98b64222 None None] Provisioning complete for port a61e4d23-b4eb-4110-a731-0d50771b3303 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:48.797647 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-044e04b3-87fd-4d5e-aeeb-5aff98b64222 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:48.805886 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:48.832922 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 0c7b8795-f4a5-424e-8b50-5a5d254488ad up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:29:48.876310 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] Set router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:29:49.046039 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:49.053718 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:49.074230 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-044e04b3-87fd-4d5e-aeeb-5aff98b64222 None None] Port a61e4d23-b4eb-4110-a731-0d50771b3303 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:29:49.080593 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d73bf983-3b61-41a9-b015-83043f782222 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:49.128144 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 0c7b8795-f4a5-424e-8b50-5a5d254488ad triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:49.128513 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:49.165243 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-533332', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-687487'] for router_interface, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:49.168943 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Pushing event updated for resources: {'Port': ['ID=a61e4d23-b4eb-4110-a731-0d50771b3303,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:49.172293 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:49.311509 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Network 496a7264-ce0a-4dd1-b29d-67154eb68a39 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:29:49.313501 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:29:49.316821 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:29:46Z', 'revision_number': 3, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:49.321046 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1d555803-87fc-414b-a20e-06d95b646e1b None None] The sync data for ids ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'] in dv-cinder-784092-9-iscsi is: [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:29:46Z', 'revision_number': 3, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:29:49.572662 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:29:49.606966 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-66e9c1a8-da87-4071-9204-60c65ce22bb4 None None] Provisioning complete for port a61e4d23-b4eb-4110-a731-0d50771b3303 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:49.607148 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-66e9c1a8-da87-4071-9204-60c65ce22bb4 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:49.681193 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-66e9c1a8-da87-4071-9204-60c65ce22bb4 None None] Port a61e4d23-b4eb-4110-a731-0d50771b3303 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:29:49.931412 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:29:49.941605 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] 10.222.0.18 "PUT /v2.0/routers/2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0/add_router_interface HTTP/1.1" status: 200 len: 503 time: 2.8577766 May 24 11:29:50.521253 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34648) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:50.526872 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Request body: {'network': {'name': 'tempest-TestShelveInstance-1223486564-network', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:50.531923 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:50.536455 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Usage tracker for resource:network and tenant:5d1d48e7fd374a08ae78e1b121cea102 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:50.539297 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:50.545017 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:50.556626 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Made reservation on behalf of 5d1d48e7fd374a08ae78e1b121cea102 for: {'network': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:29:50.558973 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-1953184'] for network, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:50.571899 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:50.586411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:50.590244 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Usage tracker for resource:security_group_rule and tenant:5d1d48e7fd374a08ae78e1b121cea102 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:50.593165 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:50.599674 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:50.699993 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:50.704045 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:50.707182 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-870969b0-bbce-4c83-84cf-719a0ca91b33 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:50.737113 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] vlan segment allocate from pool success with {'vlan_id': 500, 'physical_network': 'physnet1'} {{(pid=91819) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 24 11:29:50.746473 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=96201801-dd46-4721-be15-94e06c80399f,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:50.750369 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:50.753493 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:50.753841 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.segments_db [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Added segment f444ecab-e99d-43ff-8216-802ff480029d of type vlan for network e519e229-2b8f-4026-b816-b3ebd84b0979 May 24 11:29:50.782763 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8779718840414'] for network, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:50.827708 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774'] for network, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:50.828952 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:29:50.829500 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:50.833083 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c135b479-73e2-42f4-8ddb-2881ed393e27 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:50.846976 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 0b304a09-229c-496f-b22c-42da5ad3c052 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:29:50.922825 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 0b304a09-229c-496f-b22c-42da5ad3c052 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:29:50.928811 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 0b304a09-229c-496f-b22c-42da5ad3c052 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:50.929305 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:50.941518 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Pushing event updated for resources: {'Network': ['ID=e519e229-2b8f-4026-b816-b3ebd84b0979,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:50.947092 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] 10.222.0.18 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 0.4220612 May 24 11:29:50.947092 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-3dbbcbd7-aac6-400b-bb81-1146d944d4bf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:50.947416 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34652) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:50.953414 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Request body: {'subnet': {'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestShelveInstance-1223486564-subnet', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'ip_version': 4}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:50.958924 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Network e519e229-2b8f-4026-b816-b3ebd84b0979 details requested by agent ovs-agent-dv-cinder-784092-9-iscsi with host dv-cinder-784092-9-iscsi {{(pid=91821) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 24 11:29:50.959245 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:50.963761 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Usage tracker for resource:subnet and tenant:5d1d48e7fd374a08ae78e1b121cea102 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:50.967107 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:50.975609 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:50.989588 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Sync routers for ids ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:29:50.994523 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Made reservation on behalf of 5d1d48e7fd374a08ae78e1b121cea102 for: {'subnet': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:29:50.997290 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-535570794'] for subnet, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:51.000165 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Routers {'2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:29:51.001181 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:29:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:51.208253 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Set router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:29:51.242824 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:51.270030 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 0b304a09-229c-496f-b22c-42da5ad3c052, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=0b304a09-229c-496f-b22c-42da5ad3c052,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:29:51.274631 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:51.276982 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:51.353343 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=0b304a09-229c-496f-b22c-42da5ad3c052,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:51.358313 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:51.419695 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-534944967', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-535570812'] for subnet, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:51.423766 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:51.504595 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Pushing event updated for resources: {'Subnet': ['ID=4a49f00e-a88d-4923-829e-bbe760c82a9a,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:51.508551 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:51.515091 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Checking segment: {'id': 'f444ecab-e99d-43ff-8216-802ff480029d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:29:51.515450 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Network hostable DHCP agents. Network: e519e229-2b8f-4026-b816-b3ebd84b0979, hostable agents: ['69174cc5-06da-44a9-8bdb-046bc23ff193'], hosted agents: [] {{(pid=91819) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 24 11:29:51.534586 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Network e519e229-2b8f-4026-b816-b3ebd84b0979 is scheduled to be hosted by DHCP agent 69174cc5-06da-44a9-8bdb-046bc23ff193 with binding_index 1 {{(pid=91819) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 24 11:29:51.546661 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-784092-9-iscsi, e519e229-2b8f-4026-b816-b3ebd84b0979) {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:29:51.560579 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:29:51.561119 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:51.563083 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-9b805698-541a-4f15-8fb0-fdba9259e62d None None] Network e519e229-2b8f-4026-b816-b3ebd84b0979 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:29:51.588855 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e7560dd7-f4c0-45a0-971c-c47b462085ae admin admin] 10.222.0.18 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 0.6404049 May 24 11:29:51.592433 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34656) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:51.595295 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 4, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'a61e4d23-b4eb-4110-a731-0d50771b3303', 'name': '', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'mac_address': 'fa:16:3e:f6:4c:6d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_interface', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '86fc06b2-ed33-40b0-849c-081575fab447', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:47Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 2, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'subnets': [{'id': '86fc06b2-ed33-40b0-849c-081575fab447', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:51.599780 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Checking router: 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:29:51.600225 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Updating router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 port a61e4d23-b4eb-4110-a731-0d50771b3303 binding host dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:29:51.600991 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Request body: {'router': {'name': 'tempest-TestShelveInstance-1223486564-router', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:51.605617 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:51.610251 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Usage tracker for resource:router and tenant:5d1d48e7fd374a08ae78e1b121cea102 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:51.613175 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:51.619957 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:51.634089 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-684314', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-4157'] for router, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:51.635430 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-684570', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-687045', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-715560', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-4164'] for router, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:51.657862 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:51.696992 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:51.910255 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpea482e27-f15a-5756-a111-7e0dba035002-e519e229-2b8f-4026-b816-b3ebd84b0979', 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'fixed_ips': [{'subnet_id': '4a49f00e-a88d-4923-829e-bbe760c82a9a'}]}} from dv-cinder-784092-9-iscsi. {{(pid=91821) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 24 11:29:51.911636 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.037933 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-687072', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-684328', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-4178'] for router, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.135147 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Notify callbacks [] for port, precommit_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.148765 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.provisioning_blocks [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Transition to ACTIVE for port object a61e4d23-b4eb-4110-a731-0d50771b3303 will not be triggered until provisioned by entity L2. {{(pid=91822) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:29:52.150035 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.158653 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.163121 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.db_base_plugin_common [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Allocated IP 172.20.0.2 (e519e229-2b8f-4026-b816-b3ebd84b0979/4a49f00e-a88d-4923-829e-bbe760c82a9a/f4050d5e-51aa-4176-ac19-2a9929cf87a6) {{(pid=91821) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:29:52.164832 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-e5126f2b-bdeb-4eae-9a40-92324a393353 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:52.426342 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Notify callbacks [] for port, precommit_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.439144 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Transition to ACTIVE for port object f4050d5e-51aa-4176-ac19-2a9929cf87a6 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:29:52.442171 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Attempting to bind port a61e4d23-b4eb-4110-a731-0d50771b3303 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:29:52.442537 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Attempting to bind port a61e4d23-b4eb-4110-a731-0d50771b3303 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:29:52.442913 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Attempting to bind port a61e4d23-b4eb-4110-a731-0d50771b3303 on network 496a7264-ce0a-4dd1-b29d-67154eb68a39 {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:29:52.451193 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 29, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:29:52.451572 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Checking segment: {'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91822) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:29:52.451993 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.452388 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Bound using segment: {'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:29:52.452879 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Bound port: a61e4d23-b4eb-4110-a731-0d50771b3303, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'}}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:29:52.453288 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.464355 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Transition to ACTIVE for port object f4050d5e-51aa-4176-ac19-2a9929cf87a6 will not be triggered until provisioned by entity DHCP. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:29:52.474534 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084'] for port, after_create {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.482070 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-d37907ff-58ea-4955-971d-aa5e08408e1d None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:52.523078 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Pushing event updated for resources: {'Port': ['ID=a61e4d23-b4eb-4110-a731-0d50771b3303,revision_number=3']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:52.537164 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:52.537512 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:52.537848 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:52.538234 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:52.538566 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:52.538890 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:52.539209 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:52.539529 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:52.539876 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:52.540192 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:52.540510 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:52.542096 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91822) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:29:52.546315 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.381s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:52.578734 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] For port a61e4d23-b4eb-4110-a731-0d50771b3303, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91822) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:29:52.583775 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] For port a61e4d23-b4eb-4110-a731-0d50771b3303, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=a61e4d23-b4eb-4110-a731-0d50771b3303,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91822) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:29:52.588226 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.594168 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-32f413a7-03c7-42b6-b943-27493ecbab62 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:52.595083 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Network e519e229-2b8f-4026-b816-b3ebd84b0979 is already hosted by enough agents. {{(pid=91821) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:29:52.602458 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.scheduler.base_scheduler [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Resources bound (agent type, host, resource id): {{(pid=91821) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:29:52.605289 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Received port after_create {{(pid=91821) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:29:52.606817 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.647633 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.670936 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Routers {'2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:29:52.671569 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:29:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:52.694582 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Pushing event updated for resources: {'Port': ['ID=f4050d5e-51aa-4176-ac19-2a9929cf87a6,revision_number=1']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:52.698958 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.217s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:52.728556 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Attempting to bind port f4050d5e-51aa-4176-ac19-2a9929cf87a6 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:29:52.729105 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Attempting to bind port f4050d5e-51aa-4176-ac19-2a9929cf87a6 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'f444ecab-e99d-43ff-8216-802ff480029d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:29:52.729597 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Attempting to bind port f4050d5e-51aa-4176-ac19-2a9929cf87a6 on network e519e229-2b8f-4026-b816-b3ebd84b0979 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:29:52.737426 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 29, 26), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:29:52.737964 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Checking segment: {'id': 'f444ecab-e99d-43ff-8216-802ff480029d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:29:52.738503 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.739009 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Bound using segment: {'id': 'f444ecab-e99d-43ff-8216-802ff480029d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:29:52.739622 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Bound port: f4050d5e-51aa-4176-ac19-2a9929cf87a6, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'f444ecab-e99d-43ff-8216-802ff480029d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:29:52.740645 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.828524 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Allocated IP 172.27.0.193 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/aece7b82-6ccb-4ae5-be76-eaf6f07278b1) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:29:52.862192 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 0b304a09-229c-496f-b22c-42da5ad3c052 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:29:52.879952 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Pushing event updated for resources: {'Port': ['ID=a61e4d23-b4eb-4110-a731-0d50771b3303,revision_number=4']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:52.885577 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-5c2ae64c-61c7-4824-8606-f2d7ae0b0904 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.292s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:52.941987 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] For port f4050d5e-51aa-4176-ac19-2a9929cf87a6, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:29:52.946786 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] For port f4050d5e-51aa-4176-ac19-2a9929cf87a6, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=f4050d5e-51aa-4176-ac19-2a9929cf87a6,segment=NetworkSegment(f444ecab-e99d-43ff-8216-802ff480029d),segment_id=f444ecab-e99d-43ff-8216-802ff480029d)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:29:52.957450 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:52.963617 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:52.968723 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] Set router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:29:53.033827 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:53.040490 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 0b304a09-229c-496f-b22c-42da5ad3c052 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:53.040887 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:53.068883 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:53.074501 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:53.132979 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Pushing event updated for resources: {'Port': ['ID=f4050d5e-51aa-4176-ac19-2a9929cf87a6,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:53.145318 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.182s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:53.150468 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:29:53.153842 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:29:53.157072 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:29:53.206751 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Pushing event updated for resources: {'Port': ['ID=aece7b82-6ccb-4ae5-be76-eaf6f07278b1,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:53.210772 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.136s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:53.327296 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:29:53.373040 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 4, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:51Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'a61e4d23-b4eb-4110-a731-0d50771b3303', 'name': '', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'mac_address': 'fa:16:3e:f6:4c:6d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_interface', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '86fc06b2-ed33-40b0-849c-081575fab447', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:47Z', 'updated_at': '2021-05-24T11:29:52Z', 'revision_number': 4, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'subnets': [{'id': '86fc06b2-ed33-40b0-849c-081575fab447', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:53.380445 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c93b6cdb-6693-4b68-94fc-02df1adf155a None None] The sync data for ids ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'] in dv-cinder-784092-9-iscsi is: [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 4, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:51Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'a61e4d23-b4eb-4110-a731-0d50771b3303', 'name': '', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'mac_address': 'fa:16:3e:f6:4c:6d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_interface', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '86fc06b2-ed33-40b0-849c-081575fab447', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:47Z', 'updated_at': '2021-05-24T11:29:52Z', 'revision_number': 4, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'subnets': [{'id': '86fc06b2-ed33-40b0-849c-081575fab447', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:29:53.463938 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-97f2041e-a69b-45e2-8cd2-b213ed8dbde7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:53.700855 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:54.091327 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-684307', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-687011'] for router, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:54.158907 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-cfe7fe12-1503-4f75-8dc0-9923dbad33a7 None None] Network e519e229-2b8f-4026-b816-b3ebd84b0979 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:29:54.300542 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-4f1eaefc-8038-4e8c-ae58-85f439e24ce7 None None] Provisioning for port f4050d5e-51aa-4176-ac19-2a9929cf87a6 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:29:54.497270 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Router 2ab2d866-7255-4b75-be59-27b23905b387 is scheduled to L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 with binding_index 1 {{(pid=91819) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 24 11:29:54.692335 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:29:54.704032 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Sync routers for ids ['2ab2d866-7255-4b75-be59-27b23905b387'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:29:54.715174 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Routers {'2ab2d866-7255-4b75-be59-27b23905b387'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:29:54.716179 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:29:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2ab2d866-7255-4b75-be59-27b23905b387'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:54.721904 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] Notify callbacks [] for router, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:54.725914 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-faf320ad-5236-420e-a3ba-8c81ebb90b83 admin admin] 10.222.0.18 "POST /v2.0/routers HTTP/1.1" status: 201 len: 921 time: 3.1307731 May 24 11:29:54.726343 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34662) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:54.909785 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device a61e4d23-b4eb-4110-a731-0d50771b3303 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:29:54.989307 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Set router 2ab2d866-7255-4b75-be59-27b23905b387 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:29:55.219888 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port a61e4d23-b4eb-4110-a731-0d50771b3303 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:29:55.235311 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port a61e4d23-b4eb-4110-a731-0d50771b3303 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:55.235987 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:55.430687 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:55.476813 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2ab2d866-7255-4b75-be59-27b23905b387', 'name': 'tempest-TestShelveInstance-1223486564-router', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193'}], 'enable_snat': True}, 'gw_port_id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:51Z', 'updated_at': '2021-05-24T11:29:53Z', 'revision_number': 3, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'gw_port': {'id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:28:2e:ea', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2ab2d866-7255-4b75-be59-27b23905b387', 'device_owner': 'network:router_gateway', 'standard_attr_id': 674, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:52Z', 'updated_at': '2021-05-24T11:29:52Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:55.479850 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port a61e4d23-b4eb-4110-a731-0d50771b3303, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=a61e4d23-b4eb-4110-a731-0d50771b3303,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:29:55.484777 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Checking router: 2ab2d866-7255-4b75-be59-27b23905b387 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:29:55.485686 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Updating router 2ab2d866-7255-4b75-be59-27b23905b387 port aece7b82-6ccb-4ae5-be76-eaf6f07278b1 binding host dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:29:55.504536 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:55.515660 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:55.516516 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device f4050d5e-51aa-4176-ac19-2a9929cf87a6 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:29:55.580750 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:55.697730 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:55.750679 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=a61e4d23-b4eb-4110-a731-0d50771b3303,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:55.756669 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Allocated IP 172.20.0.1 (e519e229-2b8f-4026-b816-b3ebd84b0979/4a49f00e-a88d-4923-829e-bbe760c82a9a/4501d99c-a6db-411c-8f0c-fb483478d502) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:29:55.757616 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.178s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:55.823287 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port f4050d5e-51aa-4176-ac19-2a9929cf87a6 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:29:55.837125 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:55.841055 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port f4050d5e-51aa-4176-ac19-2a9929cf87a6 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:55.841453 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:55.914961 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:55.929915 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-13e74f2a-979b-4575-8035-4eda13003d9c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:55.931575 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Transition to ACTIVE for port object 4501d99c-a6db-411c-8f0c-fb483478d502 will not be triggered until provisioned by entity DHCP. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:29:55.961764 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:55.964771 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-8def86b3-fd8a-4900-be65-cb54f99767e2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:56.033980 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Network e519e229-2b8f-4026-b816-b3ebd84b0979 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:29:56.037022 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:29:56.040542 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:29:56.072271 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Pushing event updated for resources: {'Port': ['ID=4501d99c-a6db-411c-8f0c-fb483478d502,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:56.078852 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:56.108976 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-687570'] for router_interface, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.128380 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.225920 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port f4050d5e-51aa-4176-ac19-2a9929cf87a6, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=f4050d5e-51aa-4176-ac19-2a9929cf87a6,segment=NetworkSegment(f444ecab-e99d-43ff-8216-802ff480029d),segment_id=f444ecab-e99d-43ff-8216-802ff480029d)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:29:56.233013 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.238863 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:56.306325 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Notify callbacks [] for port, precommit_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.334355 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Transition to ACTIVE for port object aece7b82-6ccb-4ae5-be76-eaf6f07278b1 will not be triggered until provisioned by entity L2. {{(pid=91821) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:29:56.339317 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-9cb597a0-a09b-4457-b416-d75b9146fc2f None None] Provisioning for port 4501d99c-a6db-411c-8f0c-fb483478d502 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:29:56.347822 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.352626 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-9cb597a0-a09b-4457-b416-d75b9146fc2f None None] Provisioning complete for port 4501d99c-a6db-411c-8f0c-fb483478d502 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:56.353008 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-9cb597a0-a09b-4457-b416-d75b9146fc2f None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.366386 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.385346 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=f4050d5e-51aa-4176-ac19-2a9929cf87a6,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:56.388570 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.150s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:56.397696 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.045s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:56.500161 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Attempting to bind port aece7b82-6ccb-4ae5-be76-eaf6f07278b1 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:29:56.500534 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Attempting to bind port aece7b82-6ccb-4ae5-be76-eaf6f07278b1 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:29:56.500908 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Attempting to bind port aece7b82-6ccb-4ae5-be76-eaf6f07278b1 on network 858b8fad-a73f-4578-9079-6e731b9ff7e2 {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:29:56.506616 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-9cb597a0-a09b-4457-b416-d75b9146fc2f None None] Port 4501d99c-a6db-411c-8f0c-fb483478d502 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:29:56.516436 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 29, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:29:56.516822 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Checking segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91821) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:29:56.517293 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.517694 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Bound using segment: {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'} {{(pid=91821) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:29:56.518177 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.managers [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Bound port: aece7b82-6ccb-4ae5-be76-eaf6f07278b1, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'eef8fa6c-c77b-4b44-a377-fa3b8b4035b0', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2'}}] {{(pid=91821) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:29:56.518561 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.552022 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b5c97482-9d39-4ad2-a6f3-0ddeac0b161a None None] Pushing event updated for resources: {'Port': ['ID=aece7b82-6ccb-4ae5-be76-eaf6f07278b1,revision_number=2']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:56.554451 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b5c97482-9d39-4ad2-a6f3-0ddeac0b161a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.157s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:56.609746 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] For port aece7b82-6ccb-4ae5-be76-eaf6f07278b1, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91821) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:29:56.613570 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] For port aece7b82-6ccb-4ae5-be76-eaf6f07278b1, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=aece7b82-6ccb-4ae5-be76-eaf6f07278b1,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:29:56.615378 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.619259 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.620986 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.622926 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-3049afe5-bb33-40f9-8689-6286521d9591 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:56.664599 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:56.698233 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Routers {'2ab2d866-7255-4b75-be59-27b23905b387'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:29:56.698233 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:29:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2ab2d866-7255-4b75-be59-27b23905b387'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:56.747440 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-533332', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-687487'] for router_interface, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:56.755163 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b5c97482-9d39-4ad2-a6f3-0ddeac0b161a None None] Pushing event updated for resources: {'Port': ['ID=aece7b82-6ccb-4ae5-be76-eaf6f07278b1,revision_number=3']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:56.755786 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Pushing event updated for resources: {'Port': ['ID=4501d99c-a6db-411c-8f0c-fb483478d502,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:56.760094 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:56.853473 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-b5c97482-9d39-4ad2-a6f3-0ddeac0b161a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.230s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:56.855441 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device a61e4d23-b4eb-4110-a731-0d50771b3303 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:29:56.913515 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Network e519e229-2b8f-4026-b816-b3ebd84b0979 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:29:56.915023 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:29:56.976878 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port a61e4d23-b4eb-4110-a731-0d50771b3303 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:56.977426 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:57.024242 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] Set router 2ab2d866-7255-4b75-be59-27b23905b387 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:29:57.113649 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:57.145113 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Router 2ab2d866-7255-4b75-be59-27b23905b387 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:29:57.322142 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-d88ca502-cd05-496b-aec5-82b951937955 None None] Provisioning complete for port 4501d99c-a6db-411c-8f0c-fb483478d502 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:57.322764 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d88ca502-cd05-496b-aec5-82b951937955 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:57.328127 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:29:57.345497 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] 10.222.0.18 "PUT /v2.0/routers/2ab2d866-7255-4b75-be59-27b23905b387/add_router_interface HTTP/1.1" status: 200 len: 503 time: 2.6107459 May 24 11:29:57.398437 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device f4050d5e-51aa-4176-ac19-2a9929cf87a6 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:29:57.468858 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-d88ca502-cd05-496b-aec5-82b951937955 None None] Port 4501d99c-a6db-411c-8f0c-fb483478d502 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:29:57.531794 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port f4050d5e-51aa-4176-ac19-2a9929cf87a6 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:57.532179 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:57.602467 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2ab2d866-7255-4b75-be59-27b23905b387', 'name': 'tempest-TestShelveInstance-1223486564-router', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193'}], 'enable_snat': True}, 'gw_port_id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:51Z', 'updated_at': '2021-05-24T11:29:56Z', 'revision_number': 4, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'gw_port': {'id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:28:2e:ea', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2ab2d866-7255-4b75-be59-27b23905b387', 'device_owner': 'network:router_gateway', 'standard_attr_id': 674, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:52Z', 'updated_at': '2021-05-24T11:29:56Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '4501d99c-a6db-411c-8f0c-fb483478d502', 'name': '', 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'mac_address': 'fa:16:3e:89:3a:fe', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2ab2d866-7255-4b75-be59-27b23905b387', 'device_owner': 'network:router_interface', 'standard_attr_id': 675, 'fixed_ips': [{'subnet_id': '4a49f00e-a88d-4923-829e-bbe760c82a9a', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:55Z', 'updated_at': '2021-05-24T11:29:56Z', 'revision_number': 2, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'subnets': [{'id': '4a49f00e-a88d-4923-829e-bbe760c82a9a', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:29:57.610087 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1a9a2708-d8ad-4fda-8c10-485d080bbd72 None None] The sync data for ids ['2ab2d866-7255-4b75-be59-27b23905b387'] in dv-cinder-784092-9-iscsi is: [{'id': '2ab2d866-7255-4b75-be59-27b23905b387', 'name': 'tempest-TestShelveInstance-1223486564-router', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193'}], 'enable_snat': True}, 'gw_port_id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:51Z', 'updated_at': '2021-05-24T11:29:56Z', 'revision_number': 4, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'gw_port': {'id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:28:2e:ea', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2ab2d866-7255-4b75-be59-27b23905b387', 'device_owner': 'network:router_gateway', 'standard_attr_id': 674, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:52Z', 'updated_at': '2021-05-24T11:29:56Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '4501d99c-a6db-411c-8f0c-fb483478d502', 'name': '', 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'mac_address': 'fa:16:3e:89:3a:fe', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2ab2d866-7255-4b75-be59-27b23905b387', 'device_owner': 'network:router_interface', 'standard_attr_id': 675, 'fixed_ips': [{'subnet_id': '4a49f00e-a88d-4923-829e-bbe760c82a9a', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:55Z', 'updated_at': '2021-05-24T11:29:56Z', 'revision_number': 2, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'subnets': [{'id': '4a49f00e-a88d-4923-829e-bbe760c82a9a', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:29:58.081082 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34674) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:58.086543 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-1143249332', 'description': 'tempest-secgroup-smoke-1143249332 description', 'project_id': '4ee563f50afe4088aebf60ed28c59ea7'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:58.093391 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:58.098271 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Usage tracker for resource:security_group and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:58.102351 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:58.109392 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:58.123425 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.146712 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:58.151621 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Usage tracker for resource:security_group_rule and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:58.154979 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:58.161338 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:58.260194 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.266301 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.274935 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e3d507e3-64aa-42e4-823e-5ebd21c2971f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:58.320290 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event updated for resources: {'SecurityGroup': ['ID=225a506e-ecb3-47d8-9bd2-46faece5caa7,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:58.324436 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:58.325456 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.327797 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-282bcc1f-8b26-4c9b-8726-524ab7a3afe7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1794 time: 0.2445073 May 24 11:29:58.330677 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34678) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:58.336284 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-2c8c4f13-be0a-4d8b-ad8a-f241a6bbfa68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Request body: {'security_group_rule': {'security_group_id': '225a506e-ecb3-47d8-9bd2-46faece5caa7', 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:58.344070 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2c8c4f13-be0a-4d8b-ad8a-f241a6bbfa68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:58.349317 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2c8c4f13-be0a-4d8b-ad8a-f241a6bbfa68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Usage tracker for resource:security_group_rule and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:58.352597 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-2c8c4f13-be0a-4d8b-ad8a-f241a6bbfa68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:58.359464 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-2c8c4f13-be0a-4d8b-ad8a-f241a6bbfa68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:58.380563 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2c8c4f13-be0a-4d8b-ad8a-f241a6bbfa68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.468641 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2c8c4f13-be0a-4d8b-ad8a-f241a6bbfa68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.473151 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2c8c4f13-be0a-4d8b-ad8a-f241a6bbfa68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.475466 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-22857123-980d-4701-8475-f395a2e7c1da tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:58.499769 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c8c4f13-be0a-4d8b-ad8a-f241a6bbfa68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=d0cbe796-21fb-44b1-a4e7-6cfac0d21530,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:58.505111 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-2c8c4f13-be0a-4d8b-ad8a-f241a6bbfa68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:58.510889 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-2c8c4f13-be0a-4d8b-ad8a-f241a6bbfa68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.512142 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-2c8c4f13-be0a-4d8b-ad8a-f241a6bbfa68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.1804943 May 24 11:29:58.516876 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34682) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:58.522434 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-dde0807f-a44b-4fe3-b556-f2094b9e8c48 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Request body: {'security_group_rule': {'security_group_id': '225a506e-ecb3-47d8-9bd2-46faece5caa7', 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:58.528745 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-dde0807f-a44b-4fe3-b556-f2094b9e8c48 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:58.533375 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-dde0807f-a44b-4fe3-b556-f2094b9e8c48 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Usage tracker for resource:security_group_rule and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:58.536755 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-dde0807f-a44b-4fe3-b556-f2094b9e8c48 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:58.543143 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-dde0807f-a44b-4fe3-b556-f2094b9e8c48 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:58.569889 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dde0807f-a44b-4fe3-b556-f2094b9e8c48 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.643507 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dde0807f-a44b-4fe3-b556-f2094b9e8c48 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.648279 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dde0807f-a44b-4fe3-b556-f2094b9e8c48 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.654896 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:58.675702 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dde0807f-a44b-4fe3-b556-f2094b9e8c48 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=d5a00d41-1292-4cdd-82b7-31eb40526d05,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:58.678165 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-dde0807f-a44b-4fe3-b556-f2094b9e8c48 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:58.686017 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-dde0807f-a44b-4fe3-b556-f2094b9e8c48 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.687398 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-dde0807f-a44b-4fe3-b556-f2094b9e8c48 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.1694808 May 24 11:29:58.690906 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34686) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:58.696314 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-bbac0512-2708-467a-a91e-9d7e6f6b8e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Request body: {'security_group_rule': {'security_group_id': '225a506e-ecb3-47d8-9bd2-46faece5caa7', 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:58.703209 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-bbac0512-2708-467a-a91e-9d7e6f6b8e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:58.707211 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-bbac0512-2708-467a-a91e-9d7e6f6b8e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Usage tracker for resource:security_group_rule and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:58.710112 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-bbac0512-2708-467a-a91e-9d7e6f6b8e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:58.716026 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-bbac0512-2708-467a-a91e-9d7e6f6b8e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:58.732762 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bbac0512-2708-467a-a91e-9d7e6f6b8e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.803309 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bbac0512-2708-467a-a91e-9d7e6f6b8e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.807490 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bbac0512-2708-467a-a91e-9d7e6f6b8e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.809615 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-75b57414-a17b-498b-bd2a-12d8427ce274 tempest-TestMinimumBasicScenario-93741259 tempest-TestMinimumBasicScenario-93741259-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:58.833048 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device aece7b82-6ccb-4ae5-be76-eaf6f07278b1 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:29:58.835943 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bbac0512-2708-467a-a91e-9d7e6f6b8e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=27ef3c21-9fb4-4998-93f0-9a1f0cec51f5,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:58.838408 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-bbac0512-2708-467a-a91e-9d7e6f6b8e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:58.847250 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-bbac0512-2708-467a-a91e-9d7e6f6b8e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.851442 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-bbac0512-2708-467a-a91e-9d7e6f6b8e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.1569297 May 24 11:29:58.851703 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34690) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:58.857490 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-d9a864df-af1d-4a46-a32c-669eef2d45fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Request body: {'security_group_rule': {'security_group_id': '225a506e-ecb3-47d8-9bd2-46faece5caa7', 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:58.864278 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-d9a864df-af1d-4a46-a32c-669eef2d45fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:58.868396 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-d9a864df-af1d-4a46-a32c-669eef2d45fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Usage tracker for resource:security_group_rule and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:58.871435 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-d9a864df-af1d-4a46-a32c-669eef2d45fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:58.877795 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-d9a864df-af1d-4a46-a32c-669eef2d45fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:58.900650 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d9a864df-af1d-4a46-a32c-669eef2d45fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.912256 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port aece7b82-6ccb-4ae5-be76-eaf6f07278b1 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:29:58.919938 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port aece7b82-6ccb-4ae5-be76-eaf6f07278b1 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:29:58.920408 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.984922 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d9a864df-af1d-4a46-a32c-669eef2d45fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.987554 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.990044 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d9a864df-af1d-4a46-a32c-669eef2d45fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:58.992297 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:59.022958 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d9a864df-af1d-4a46-a32c-669eef2d45fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=2038edac-1b39-43be-afde-0e117833824f,revision_number=0']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:59.026098 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d9a864df-af1d-4a46-a32c-669eef2d45fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:59.027977 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-d9a864df-af1d-4a46-a32c-669eef2d45fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:59.030264 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-d9a864df-af1d-4a46-a32c-669eef2d45fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.1773105 May 24 11:29:59.033656 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 34694) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:59.039765 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Request body: {'security_group_rule': {'security_group_id': '225a506e-ecb3-47d8-9bd2-46faece5caa7', 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:59.048661 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:59.055486 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Usage tracker for resource:security_group_rule and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:59.066340 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:59.067758 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:59.076670 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:59.085893 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port aece7b82-6ccb-4ae5-be76-eaf6f07278b1, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=aece7b82-6ccb-4ae5-be76-eaf6f07278b1,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:29:59.090020 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:59.091984 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-c285c667-1f23-4119-b92b-c1cf20d046f7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:59.092533 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.securitygroups_db [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Project 4ee563f50afe4088aebf60ed28c59ea7 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 24 11:29:59.100117 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:59.164350 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=aece7b82-6ccb-4ae5-be76-eaf6f07278b1,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:59.167319 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:59.171533 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:59.176831 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:59.179709 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-9132029b-9b77-4baa-b506-53a2de318231 tempest-AttachVolumeShelveTestJSON-1093338786 tempest-AttachVolumeShelveTestJSON-1093338786-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:59.202064 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=0df8846c-4860-44c2-b092-386557c42384,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:59.205606 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:59.213302 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:59.214785 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-0dae4091-4af2-491e-bfc7-312add22c5e6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.1800172 May 24 11:29:59.218758 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 34698) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:29:59.225220 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.v2.base [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Request body: {'security_group_rule': {'security_group_id': '225a506e-ecb3-47d8-9bd2-46faece5caa7', 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=91820) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:29:59.231482 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:29:59.235809 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Usage tracker for resource:security_group_rule and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:29:59.239349 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.quota.resource [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=91820) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:29:59.247559 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.db.quota.driver [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=91820) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:29:59.261117 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.db.securitygroups_db [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Project 4ee563f50afe4088aebf60ed28c59ea7 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 24 11:29:59.268082 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:59.338990 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:59.343043 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-711723'] for security_group_rule, after_create {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:59.349987 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:29:59.374353 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=ed585d01-ae50-4692-80e4-83e169052df6,revision_number=0']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:29:59.379378 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:29:59.384743 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:29:59.386297 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-c632ec74-86c2-417d-9add-98b57b07b0ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.1663766 May 24 11:30:00.821231 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device aece7b82-6ccb-4ae5-be76-eaf6f07278b1 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:30:00.902553 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port aece7b82-6ccb-4ae5-be76-eaf6f07278b1 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:30:00.902956 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:01.168689 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Sync routers for ids ['2ab2d866-7255-4b75-be59-27b23905b387'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:30:01.179327 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Routers {'2ab2d866-7255-4b75-be59-27b23905b387'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:30:01.180342 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:29:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2ab2d866-7255-4b75-be59-27b23905b387'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:30:01.395542 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Set router 2ab2d866-7255-4b75-be59-27b23905b387 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:30:01.737616 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2ab2d866-7255-4b75-be59-27b23905b387', 'name': 'tempest-TestShelveInstance-1223486564-router', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193'}], 'enable_snat': True}, 'gw_port_id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:51Z', 'updated_at': '2021-05-24T11:29:56Z', 'revision_number': 4, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'gw_port': {'id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:28:2e:ea', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2ab2d866-7255-4b75-be59-27b23905b387', 'device_owner': 'network:router_gateway', 'standard_attr_id': 674, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:52Z', 'updated_at': '2021-05-24T11:29:59Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '4501d99c-a6db-411c-8f0c-fb483478d502', 'name': '', 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'mac_address': 'fa:16:3e:89:3a:fe', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2ab2d866-7255-4b75-be59-27b23905b387', 'device_owner': 'network:router_interface', 'standard_attr_id': 675, 'fixed_ips': [{'subnet_id': '4a49f00e-a88d-4923-829e-bbe760c82a9a', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:55Z', 'updated_at': '2021-05-24T11:29:56Z', 'revision_number': 2, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'subnets': [{'id': '4a49f00e-a88d-4923-829e-bbe760c82a9a', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:30:01.741647 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Checking router: 2ab2d866-7255-4b75-be59-27b23905b387 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:30:01.742001 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Updating router 2ab2d866-7255-4b75-be59-27b23905b387 port 4501d99c-a6db-411c-8f0c-fb483478d502 binding host dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 24 11:30:01.828508 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:02.179441 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Notify callbacks [] for port, precommit_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:02.191770 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.provisioning_blocks [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Transition to ACTIVE for port object 4501d99c-a6db-411c-8f0c-fb483478d502 will not be triggered until provisioned by entity L2. {{(pid=91822) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:30:02.198681 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:02.202807 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-52070a7e-091b-40cb-b656-050f232a2091 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:30:02.395890 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98d801d9-f5a5-4622-8c6c-9a668dffcf10 None None] Pushing event updated for resources: {'Port': ['ID=4501d99c-a6db-411c-8f0c-fb483478d502,revision_number=3']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:30:02.404156 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-98d801d9-f5a5-4622-8c6c-9a668dffcf10 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.200s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:30:02.462853 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Attempting to bind port 4501d99c-a6db-411c-8f0c-fb483478d502 on host dv-cinder-784092-9-iscsi for vnic_type normal with profile {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:30:02.463236 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Attempting to bind port 4501d99c-a6db-411c-8f0c-fb483478d502 by drivers openvswitch,hyperv on host dv-cinder-784092-9-iscsi at level 0 using segments [{'id': 'f444ecab-e99d-43ff-8216-802ff480029d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979'}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:30:02.463618 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Attempting to bind port 4501d99c-a6db-411c-8f0c-fb483478d502 on network e519e229-2b8f-4026-b816-b3ebd84b0979 {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:30:02.469575 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Checking agent: {'id': 'd9c3b179-b48e-4ac6-9ab3-a727750ac51d', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-784092-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 7, 56), 'started_at': datetime.datetime(2021, 5, 24, 11, 8, 26), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 29, 56), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-784092-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:30:02.470004 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Checking segment: {'id': 'f444ecab-e99d-43ff-8216-802ff480029d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91822) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:30:02.470614 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:02.471024 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Bound using segment: {'id': 'f444ecab-e99d-43ff-8216-802ff480029d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979'} {{(pid=91822) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:30:02.471539 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.managers [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Bound port: 4501d99c-a6db-411c-8f0c-fb483478d502, host: dv-cinder-784092-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'f444ecab-e99d-43ff-8216-802ff480029d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 500, 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979'}}] {{(pid=91822) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:30:02.471942 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:02.589123 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] For port 4501d99c-a6db-411c-8f0c-fb483478d502, host dv-cinder-784092-9-iscsi, cleared binding levels {{(pid=91822) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:30:02.592183 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.plugins.ml2.db [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] For port 4501d99c-a6db-411c-8f0c-fb483478d502, host dv-cinder-784092-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=4501d99c-a6db-411c-8f0c-fb483478d502,segment=NetworkSegment(f444ecab-e99d-43ff-8216-802ff480029d),segment_id=f444ecab-e99d-43ff-8216-802ff480029d)] {{(pid=91822) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:30:02.598256 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron_lib.callbacks.manager [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91822) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:02.602531 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-96e21e33-f197-465c-8afc-10484f574a70 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:30:02.703371 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Routers {'2ab2d866-7255-4b75-be59-27b23905b387'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:30:02.704024 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:29:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2ab2d866-7255-4b75-be59-27b23905b387'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:30:02.831466 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98d801d9-f5a5-4622-8c6c-9a668dffcf10 None None] Pushing event updated for resources: {'Port': ['ID=4501d99c-a6db-411c-8f0c-fb483478d502,revision_number=4']} {{(pid=91822) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:30:02.842924 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG oslo_concurrency.lockutils [None req-98d801d9-f5a5-4622-8c6c-9a668dffcf10 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.240s {{(pid=91822) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:30:02.898286 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] Set router 2ab2d866-7255-4b75-be59-27b23905b387 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:30:03.212313 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2ab2d866-7255-4b75-be59-27b23905b387', 'name': 'tempest-TestShelveInstance-1223486564-router', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193'}], 'enable_snat': True}, 'gw_port_id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:51Z', 'updated_at': '2021-05-24T11:29:56Z', 'revision_number': 4, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'gw_port': {'id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:28:2e:ea', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2ab2d866-7255-4b75-be59-27b23905b387', 'device_owner': 'network:router_gateway', 'standard_attr_id': 674, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:52Z', 'updated_at': '2021-05-24T11:29:59Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '4501d99c-a6db-411c-8f0c-fb483478d502', 'name': '', 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'mac_address': 'fa:16:3e:89:3a:fe', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2ab2d866-7255-4b75-be59-27b23905b387', 'device_owner': 'network:router_interface', 'standard_attr_id': 675, 'fixed_ips': [{'subnet_id': '4a49f00e-a88d-4923-829e-bbe760c82a9a', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:55Z', 'updated_at': '2021-05-24T11:30:02Z', 'revision_number': 4, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'subnets': [{'id': '4a49f00e-a88d-4923-829e-bbe760c82a9a', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:30:03.216257 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f68cbb35-12e3-4056-acd1-053ff1a3e029 None None] The sync data for ids ['2ab2d866-7255-4b75-be59-27b23905b387'] in dv-cinder-784092-9-iscsi is: [{'id': '2ab2d866-7255-4b75-be59-27b23905b387', 'name': 'tempest-TestShelveInstance-1223486564-router', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193'}], 'enable_snat': True}, 'gw_port_id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:51Z', 'updated_at': '2021-05-24T11:29:56Z', 'revision_number': 4, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'gw_port': {'id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:28:2e:ea', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2ab2d866-7255-4b75-be59-27b23905b387', 'device_owner': 'network:router_gateway', 'standard_attr_id': 674, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:52Z', 'updated_at': '2021-05-24T11:29:59Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': '4501d99c-a6db-411c-8f0c-fb483478d502', 'name': '', 'network_id': 'e519e229-2b8f-4026-b816-b3ebd84b0979', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'mac_address': 'fa:16:3e:89:3a:fe', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2ab2d866-7255-4b75-be59-27b23905b387', 'device_owner': 'network:router_interface', 'standard_attr_id': 675, 'fixed_ips': [{'subnet_id': '4a49f00e-a88d-4923-829e-bbe760c82a9a', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:55Z', 'updated_at': '2021-05-24T11:30:02Z', 'revision_number': 4, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'subnets': [{'id': '4a49f00e-a88d-4923-829e-bbe760c82a9a', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:30:04.825324 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 4501d99c-a6db-411c-8f0c-fb483478d502 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:30:04.891385 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning for port 4501d99c-a6db-411c-8f0c-fb483478d502 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:30:04.896075 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 4501d99c-a6db-411c-8f0c-fb483478d502 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:30:04.896477 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:05.014279 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:05.045885 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] For port 4501d99c-a6db-411c-8f0c-fb483478d502, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=4501d99c-a6db-411c-8f0c-fb483478d502,segment=NetworkSegment(f444ecab-e99d-43ff-8216-802ff480029d),segment_id=f444ecab-e99d-43ff-8216-802ff480029d)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:30:05.051441 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:05.054728 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:30:05.122774 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Pushing event updated for resources: {'Port': ['ID=4501d99c-a6db-411c-8f0c-fb483478d502,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:30:05.126896 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:30:06.808087 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 4501d99c-a6db-411c-8f0c-fb483478d502 up at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:30:06.876548 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Provisioning complete for port 4501d99c-a6db-411c-8f0c-fb483478d502 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:30:06.877015 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:15.652550 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-8d9603f0-7d5f-49a7-9630-0813110e98f4 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-1143249332&fields=id&tenant_id=4ee563f50afe4088aebf60ed28c59ea7 HTTP/1.1" status: 200 len: 267 time: 0.0408618 May 24 11:30:15.926041 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-171b68e4-aa14-482f-8c68-e0f06e328b6a tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:15.927413 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-171b68e4-aa14-482f-8c68-e0f06e328b6a tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/networks?id=496a7264-ce0a-4dd1-b29d-67154eb68a39 HTTP/1.1" status: 200 len: 808 time: 0.2672584 May 24 11:30:15.942607 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-64d09319-0773-414c-a462-b849257ade08 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/quotas/4ee563f50afe4088aebf60ed28c59ea7 HTTP/1.1" status: 200 len: 375 time: 0.0095887 May 24 11:30:16.029242 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-981e1235-14e7-4eb2-951a-f5043a82dcc0 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?tenant_id=4ee563f50afe4088aebf60ed28c59ea7&fields=id HTTP/1.1" status: 200 len: 302 time: 0.0808094 May 24 11:30:16.170141 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-b62eabc6-d912-4717-bc5a-d78b70fd977b service neutron] 10.222.0.18 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1289160 May 24 11:30:16.304775 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-f0c5b24a-39a7-4cc5-a222-c12b680588c1 service neutron] 10.222.0.18 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1284864 May 24 11:30:16.711108 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-73fcec51-2eb1-41c1-ba5f-6f71be5d2733 req-dd2169a4-f4ee-4309-8510-5c32607e8e98 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Synchronizing usage tracker for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:30:16.733222 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-73fcec51-2eb1-41c1-ba5f-6f71be5d2733 req-dd2169a4-f4ee-4309-8510-5c32607e8e98 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Unset dirty status for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:30:16.734642 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-73fcec51-2eb1-41c1-ba5f-6f71be5d2733 req-dd2169a4-f4ee-4309-8510-5c32607e8e98 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 210 time: 0.0676467 May 24 11:30:17.790585 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-06632577-7d5e-43a2-903b-d98ca676f911 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:17.791378 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-06632577-7d5e-43a2-903b-d98ca676f911 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Synchronizing usage tracker for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:network {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:30:17.814830 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-06632577-7d5e-43a2-903b-d98ca676f911 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Unset dirty status for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:network {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:30:17.816341 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-06632577-7d5e-43a2-903b-d98ca676f911 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.93 "GET /v2.0/networks?id=496a7264-ce0a-4dd1-b29d-67154eb68a39 HTTP/1.1" status: 200 len: 808 time: 0.1849806 May 24 11:30:17.879409 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-b2ae2692-0f34-4829-8604-fbe6892d1603 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Synchronizing usage tracker for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:security_group {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:30:17.900568 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-b2ae2692-0f34-4829-8604-fbe6892d1603 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Unset dirty status for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:security_group {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:30:17.902074 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-b2ae2692-0f34-4829-8604-fbe6892d1603 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.93 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=4ee563f50afe4088aebf60ed28c59ea7 HTTP/1.1" status: 200 len: 380 time: 0.0776584 May 24 11:30:17.902980 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3cd187ec-b864-4070-8c49-060a3c5bf6ae req-a1ce0a15-b86e-4c77-b66e-e18399032365 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 210 time: 0.0531142 May 24 11:30:17.911169 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Request body: {'port': {'device_id': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'admin_state_up': True, 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'security_groups': ['225a506e-ecb3-47d8-9bd2-46faece5caa7']}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:30:18.176125 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:30:18.186613 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:30:18.197362 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Made reservation on behalf of 4ee563f50afe4088aebf60ed28c59ea7 for: {'port': 1} {{(pid=91819) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 24 11:30:18.200987 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:18.382089 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Allocated IP 172.20.0.11 (496a7264-ce0a-4dd1-b29d-67154eb68a39/86fc06b2-ed33-40b0-849c-081575fab447/66674db0-9a62-4491-be8b-38bf92ccfa58) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:30:18.586873 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:18.602837 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Transition to ACTIVE for port object 66674db0-9a62-4491-be8b-38bf92ccfa58 will not be triggered until provisioned by entity DHCP. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:30:18.627864 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:18.635107 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c8ba11ff-4d58-4ffa-8601-97b748052e6b service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:30:18.700559 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Network 496a7264-ce0a-4dd1-b29d-67154eb68a39 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:30:18.704014 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:30:18.706769 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:30:18.751730 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event updated for resources: {'Port': ['ID=66674db0-9a62-4491-be8b-38bf92ccfa58,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:30:18.755313 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.120s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:30:18.789377 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:18.790348 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:18.817205 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.93 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 0.9099112 May 24 11:30:18.829129 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Request body: {'port': {'device_id': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78409'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:30:18.947305 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:18.974678 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-94d163cb-547b-4997-b6b9-219e1f484b7d None None] Provisioning for port 66674db0-9a62-4491-be8b-38bf92ccfa58 completed by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:30:18.981846 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-94d163cb-547b-4997-b6b9-219e1f484b7d None None] Provisioning complete for port 66674db0-9a62-4491-be8b-38bf92ccfa58 triggered by entity DHCP. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:30:18.982815 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-94d163cb-547b-4997-b6b9-219e1f484b7d None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:19.042915 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-94d163cb-547b-4997-b6b9-219e1f484b7d None None] Port 66674db0-9a62-4491-be8b-38bf92ccfa58 cannot update to ACTIVE because it is not bound. {{(pid=91821) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 24 11:30:19.264464 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-26d18e36-1e30-4401-831f-6e929e05270b req-7f72e5ad-4d5b-46c7-adde-3c0411a18120 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:19.265183 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-26d18e36-1e30-4401-831f-6e929e05270b req-7f72e5ad-4d5b-46c7-adde-3c0411a18120 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Synchronizing usage tracker for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:30:19.310066 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-26d18e36-1e30-4401-831f-6e929e05270b req-7f72e5ad-4d5b-46c7-adde-3c0411a18120 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Unset dirty status for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:30:19.311778 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-26d18e36-1e30-4401-831f-6e929e05270b req-7f72e5ad-4d5b-46c7-adde-3c0411a18120 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 932 time: 0.2925766 May 24 11:30:19.389030 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-26d18e36-1e30-4401-831f-6e929e05270b req-f870408e-e6eb-400b-9e40-8982374de4fc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0610168 May 24 11:30:19.459787 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 66674db0-9a62-4491-be8b-38bf92ccfa58 {{(pid=91819) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:30:19.467252 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:19.481339 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.provisioning_blocks [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Transition to ACTIVE for port object 66674db0-9a62-4491-be8b-38bf92ccfa58 will not be triggered until provisioned by entity L2. {{(pid=91819) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 24 11:30:19.486385 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:19.490128 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-6f403013-1b17-40dc-b934-2883102cded4 tempest-VolumesSnapshotTestJSON-1040878925 tempest-VolumesSnapshotTestJSON-1040878925-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:30:19.661439 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Pushing event updated for resources: {'Port': ['ID=66674db0-9a62-4491-be8b-38bf92ccfa58,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:30:19.670397 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.180s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:30:19.772879 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Attempting to bind port 66674db0-9a62-4491-be8b-38bf92ccfa58 on host hv-cinder-78409 for vnic_type normal with profile {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 24 11:30:19.773241 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Attempting to bind port 66674db0-9a62-4491-be8b-38bf92ccfa58 by drivers openvswitch,hyperv on host hv-cinder-78409 at level 0 using segments [{'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 24 11:30:19.773568 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Attempting to bind port 66674db0-9a62-4491-be8b-38bf92ccfa58 on network 496a7264-ce0a-4dd1-b29d-67154eb68a39 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:30:19.778833 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Port 66674db0-9a62-4491-be8b-38bf92ccfa58 on network 496a7264-ce0a-4dd1-b29d-67154eb68a39 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78409 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 24 11:30:19.779151 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Attempting to bind port 66674db0-9a62-4491-be8b-38bf92ccfa58 on network 496a7264-ce0a-4dd1-b29d-67154eb68a39 {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 24 11:30:19.787051 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Checking agent: {'id': '754667ed-1b92-4d3f-bf4a-b156105c033e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78409', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'started_at': datetime.datetime(2021, 5, 24, 11, 14, 21), 'heartbeat_timestamp': datetime.datetime(2021, 5, 24, 11, 29, 52), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 24 11:30:19.787384 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Checking segment: {'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91819) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 24 11:30:19.787766 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Bound using segment: {'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'} {{(pid=91819) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 24 11:30:19.788193 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.managers [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Bound port: 66674db0-9a62-4491-be8b-38bf92ccfa58, host: hv-cinder-78409, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '417bf2a8-a5cf-497b-a631-e0f581048962', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 540, 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39'}}] {{(pid=91819) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 24 11:30:19.788536 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:19.883798 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] For port 66674db0-9a62-4491-be8b-38bf92ccfa58, host hv-cinder-78409, cleared binding levels {{(pid=91819) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 24 11:30:19.887040 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] For port 66674db0-9a62-4491-be8b-38bf92ccfa58, host hv-cinder-78409, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=66674db0-9a62-4491-be8b-38bf92ccfa58,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91819) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 24 11:30:19.892545 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:19.896253 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c8bf525c-6fa4-4f7e-a9a2-017a1c1754b2 tempest-TestVolumeBootPattern-1657758896 tempest-TestVolumeBootPattern-1657758896-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:30:19.958857 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:19.959460 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:19.960804 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] 10.222.0.93 "PUT /v2.0/ports/66674db0-9a62-4491-be8b-38bf92ccfa58 HTTP/1.1" status: 200 len: 1094 time: 1.1366086 May 24 11:30:19.961336 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:30:19.985753 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Pushing event updated for resources: {'Port': ['ID=66674db0-9a62-4491-be8b-38bf92ccfa58,revision_number=3']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:30:19.994497 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:30:20.166094 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'status': 'completed', 'code': 200} May 24 11:30:20.211012 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-9d5dda15-a162-4709-ac1d-6f4a3471b774 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:20.212471 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-9d5dda15-a162-4709-ac1d-6f4a3471b774 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=4ee563f50afe4088aebf60ed28c59ea7&device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 1097 time: 0.2072351 May 24 11:30:20.286335 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-e5a1a9f3-1dfa-46cb-9548-b689e59c1933 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=66674db0-9a62-4491-be8b-38bf92ccfa58 HTTP/1.1" status: 200 len: 217 time: 0.0659544 May 24 11:30:20.349416 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-b5552608-9988-4b8d-b51f-358b4e2313f3 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:20.350850 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-b5552608-9988-4b8d-b51f-358b4e2313f3 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=86fc06b2-ed33-40b0-849c-081575fab447 HTTP/1.1" status: 200 len: 847 time: 0.0591967 May 24 11:30:20.434751 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-30264868-cb57-4184-8e4c-5dd42e0d4117 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:20.435961 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-30264868-cb57-4184-8e4c-5dd42e0d4117 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=496a7264-ce0a-4dd1-b29d-67154eb68a39&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0793681 May 24 11:30:20.626905 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-83b85148-c411-44b9-bd62-5051c8d841b6 service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1838362 May 24 11:30:20.662064 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25148e01-2c58-4632-b532-467d844e1540 req-01dc89b0-4053-43fd-b6d3-b2be5d2e1bed tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:20.663601 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-25148e01-2c58-4632-b532-467d844e1540 req-01dc89b0-4053-43fd-b6d3-b2be5d2e1bed tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.1552856 May 24 11:30:20.786345 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-25148e01-2c58-4632-b532-467d844e1540 req-7a97663a-fbc1-451c-95b1-9ef2841afab8 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0759616 May 24 11:30:20.809326 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-826b56d6-d642-4ae9-948d-87d76657f4ab req-44487544-6737-4388-8eb5-1c2d2195338c service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1755135 May 24 11:30:20.941659 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-14af1482-f90e-45fa-ae8e-1538a0f1dc85 req-39d4e4a4-844b-4aab-a908-d897faee4858 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:20.943246 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-14af1482-f90e-45fa-ae8e-1538a0f1dc85 req-39d4e4a4-844b-4aab-a908-d897faee4858 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=4ee563f50afe4088aebf60ed28c59ea7&device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 1097 time: 0.0808039 May 24 11:30:21.079139 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-14af1482-f90e-45fa-ae8e-1538a0f1dc85 req-1db664d8-be2e-4673-89ab-ed73453fc17e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:21.080578 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-14af1482-f90e-45fa-ae8e-1538a0f1dc85 req-1db664d8-be2e-4673-89ab-ed73453fc17e service neutron] 10.222.0.93 "GET /v2.0/networks?id=496a7264-ce0a-4dd1-b29d-67154eb68a39 HTTP/1.1" status: 200 len: 909 time: 0.1305792 May 24 11:30:21.148446 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-14af1482-f90e-45fa-ae8e-1538a0f1dc85 req-b0148dd6-0c7b-4dbf-897e-bc535369bd2b service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=66674db0-9a62-4491-be8b-38bf92ccfa58 HTTP/1.1" status: 200 len: 217 time: 0.0612762 May 24 11:30:21.220098 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-14af1482-f90e-45fa-ae8e-1538a0f1dc85 req-9e33b119-60f9-4f20-b212-992c3e899e5b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:21.221582 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-14af1482-f90e-45fa-ae8e-1538a0f1dc85 req-9e33b119-60f9-4f20-b212-992c3e899e5b service neutron] 10.222.0.93 "GET /v2.0/subnets?id=86fc06b2-ed33-40b0-849c-081575fab447 HTTP/1.1" status: 200 len: 847 time: 0.0670843 May 24 11:30:21.305502 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-14af1482-f90e-45fa-ae8e-1538a0f1dc85 req-9706499b-0558-4747-849c-ae536febf70d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:21.306980 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-14af1482-f90e-45fa-ae8e-1538a0f1dc85 req-9706499b-0558-4747-849c-ae536febf70d service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=496a7264-ce0a-4dd1-b29d-67154eb68a39&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0793104 May 24 11:30:21.444491 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-14af1482-f90e-45fa-ae8e-1538a0f1dc85 req-6e5dca34-f843-4135-8412-a2b7033fd7fe service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1305311 May 24 11:30:21.584058 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-14af1482-f90e-45fa-ae8e-1538a0f1dc85 req-ab92a175-db54-4786-86ba-2ca79ca0c01b service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1333785 May 24 11:30:21.995980 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f383949-cf45-4bca-b3fc-00a44b40b298 req-6220c3e4-6482-498e-903f-55e586b5446a tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:21.996648 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7f383949-cf45-4bca-b3fc-00a44b40b298 req-6220c3e4-6482-498e-903f-55e586b5446a tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0954587 May 24 11:30:22.047957 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7f383949-cf45-4bca-b3fc-00a44b40b298 req-6293b1c1-50c7-4fed-b7a2-c03d8f53157c tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0429261 May 24 11:30:22.128788 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:23.244203 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ead88c92-0de2-4005-9038-dadac33062b6 req-cdd0cb5c-1bb8-4cf6-b3ed-8cb56096aa5b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:23.245759 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ead88c92-0de2-4005-9038-dadac33062b6 req-cdd0cb5c-1bb8-4cf6-b3ed-8cb56096aa5b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0887027 May 24 11:30:23.292364 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-ead88c92-0de2-4005-9038-dadac33062b6 req-9262f759-c617-4f20-b352-2fcf14d3ae03 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0402520 May 24 11:30:23.458222 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-10492217-4efb-4bb3-8cfa-040774f35063 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:24.486980 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca90c6b0-151f-47fb-825f-da21b4329cef req-b5c7c205-860b-4ede-a8eb-8392bde8ab33 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:24.488528 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ca90c6b0-151f-47fb-825f-da21b4329cef req-b5c7c205-860b-4ede-a8eb-8392bde8ab33 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0892444 May 24 11:30:24.541547 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ca90c6b0-151f-47fb-825f-da21b4329cef req-8a129654-9f58-4b4e-a2d8-ab5eb15722dc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0456831 May 24 11:30:25.752411 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-537b7347-cc5e-4c21-a663-08925bdbac10 req-3ca317e1-d068-4b21-b7cb-02b3f90a5f8a tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:25.754299 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-537b7347-cc5e-4c21-a663-08925bdbac10 req-3ca317e1-d068-4b21-b7cb-02b3f90a5f8a tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0921614 May 24 11:30:25.785404 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:25.802875 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-537b7347-cc5e-4c21-a663-08925bdbac10 req-a4622c20-6e54-4b7e-87be-5e817e32ba57 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0432258 May 24 11:30:25.894074 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-55e3ceb7-c8a6-48f4-bd3a-0e13be1efad1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:27.000446 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b063bb34-2c81-4409-884b-61a684511ca1 req-7fe2069b-b411-4bfc-8446-96ab5dd219ed tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:27.001957 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b063bb34-2c81-4409-884b-61a684511ca1 req-7fe2069b-b411-4bfc-8446-96ab5dd219ed tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0831463 May 24 11:30:27.029363 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:27.050382 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b063bb34-2c81-4409-884b-61a684511ca1 req-f5a611ab-22b2-40a6-a4e6-f7ef9a855608 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0425971 May 24 11:30:28.354485 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08614c52-0e43-425d-96c7-3dc5cfb181bf req-1f81bd38-2a70-4940-8fc4-e87e9ce7fc0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:28.356794 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-08614c52-0e43-425d-96c7-3dc5cfb181bf req-1f81bd38-2a70-4940-8fc4-e87e9ce7fc0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.1993310 May 24 11:30:28.403619 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-08614c52-0e43-425d-96c7-3dc5cfb181bf req-e60f3e56-6bbf-496c-adc5-8c094ef95214 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0415208 May 24 11:30:28.943753 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 66674db0-9a62-4491-be8b-38bf92ccfa58 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:30:29.026320 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:29.092356 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 66674db0-9a62-4491-be8b-38bf92ccfa58, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=66674db0-9a62-4491-be8b-38bf92ccfa58,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:30:29.106910 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'port_id': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'mac_address': 'fa:16:3e:8b:76:f6', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 540, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '86fc06b2-ed33-40b0-849c-081575fab447', 'ip_address': '172.20.0.11'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:30:29.165595 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:29.183566 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 66674db0-9a62-4491-be8b-38bf92ccfa58 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:30:29.203751 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 66674db0-9a62-4491-be8b-38bf92ccfa58, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=66674db0-9a62-4491-be8b-38bf92ccfa58,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:30:29.210091 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:29.211871 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:30:29.285643 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=66674db0-9a62-4491-be8b-38bf92ccfa58,revision_number=4']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:30:29.288675 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:30:29.413503 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['66674db0-9a62-4491-be8b-38bf92ccfa58'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:30:29.647253 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e56655f0-377f-47e1-a06d-d3abce554cf8 req-29f2d050-f7a2-460e-8120-989062c10679 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:29.648912 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e56655f0-377f-47e1-a06d-d3abce554cf8 req-29f2d050-f7a2-460e-8120-989062c10679 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 945 time: 0.1233206 May 24 11:30:29.703017 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e56655f0-377f-47e1-a06d-d3abce554cf8 req-a6d2c4bd-a4c1-4533-9634-1be589a129f2 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0466468 May 24 11:30:30.306435 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:30:30.900779 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7e39c93f-c3d3-4ab2-9a48-5e8b4b08aa23 req-cb21f8d8-9bd4-4bdd-a1d2-56288ec906e2 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:30.902268 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7e39c93f-c3d3-4ab2-9a48-5e8b4b08aa23 req-cb21f8d8-9bd4-4bdd-a1d2-56288ec906e2 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 945 time: 0.0855205 May 24 11:30:30.942961 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 66674db0-9a62-4491-be8b-38bf92ccfa58 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:30:30.951629 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7e39c93f-c3d3-4ab2-9a48-5e8b4b08aa23 req-b54144ed-ffae-4bfd-a225-620c0bd9851a tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0425529 May 24 11:30:31.016169 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning for port 66674db0-9a62-4491-be8b-38bf92ccfa58 completed by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 24 11:30:31.020943 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 66674db0-9a62-4491-be8b-38bf92ccfa58 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:30:31.021358 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:31.140184 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:31.168907 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:30:31.179414 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 66674db0-9a62-4491-be8b-38bf92ccfa58, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=66674db0-9a62-4491-be8b-38bf92ccfa58,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:30:31.183557 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:31.185853 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-f13c6e44-2fe0-409a-971d-12b002fd70b8 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:30:31.217877 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'code': 200} May 24 11:30:31.258425 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=66674db0-9a62-4491-be8b-38bf92ccfa58,revision_number=5']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:30:31.261351 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:30:31.491269 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-65803e47-9940-4d21-9616-4fd5c8776fe7 req-75cbfe04-8428-4e78-ab71-78beff8a26d5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:31.492644 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-65803e47-9940-4d21-9616-4fd5c8776fe7 req-75cbfe04-8428-4e78-ab71-78beff8a26d5 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 1099 time: 0.0726762 May 24 11:30:31.535332 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-65803e47-9940-4d21-9616-4fd5c8776fe7 req-e92641d1-b8ef-42ec-90a0-2ea9b8b62119 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0366220 May 24 11:30:32.157589 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3aa9d738-96a4-425c-aadc-33b66a2de4fa req-80c7b753-f2c4-48f7-90c4-4e9fa2e8fc1d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:32.159083 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3aa9d738-96a4-425c-aadc-33b66a2de4fa req-80c7b753-f2c4-48f7-90c4-4e9fa2e8fc1d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.1015217 May 24 11:30:32.212131 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-3aa9d738-96a4-425c-aadc-33b66a2de4fa req-8701b4f9-0efa-42fc-9409-f4acd60c347d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0437529 May 24 11:30:33.404147 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7017e841-c04b-4e3c-a3d5-312a2bd06f7d req-b4153a6f-a7f4-4b27-9fca-5325fc0e4e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:33.405667 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7017e841-c04b-4e3c-a3d5-312a2bd06f7d req-b4153a6f-a7f4-4b27-9fca-5325fc0e4e1f tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0810599 May 24 11:30:33.454388 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-7017e841-c04b-4e3c-a3d5-312a2bd06f7d req-2f1194b9-d71d-4c8d-976b-bb16c4ee01b4 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0423059 May 24 11:30:33.660129 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6a978758-7efc-4c2f-9816-6a1490ff44d7 req-97f39a5d-5820-42b3-8494-e34c71905b86 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:33.661641 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6a978758-7efc-4c2f-9816-6a1490ff44d7 req-97f39a5d-5820-42b3-8494-e34c71905b86 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0837369 May 24 11:30:33.711419 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6a978758-7efc-4c2f-9816-6a1490ff44d7 req-69c406e4-612f-4eca-84b0-7c7487ecf60b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0440435 May 24 11:30:33.735875 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34850) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:30:33.931480 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ca43ebdd-60f5-42f8-8596-1f369151d583 tempest-TestShelveInstance-1915169858 tempest-TestShelveInstance-1915169858-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:33.932178 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-ca43ebdd-60f5-42f8-8596-1f369151d583 tempest-TestShelveInstance-1915169858 tempest-TestShelveInstance-1915169858-project-admin] Synchronizing usage tracker for tenant:5d1d48e7fd374a08ae78e1b121cea102 on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:30:33.948761 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-ca43ebdd-60f5-42f8-8596-1f369151d583 tempest-TestShelveInstance-1915169858 tempest-TestShelveInstance-1915169858-project-admin] Unset dirty status for tenant:5d1d48e7fd374a08ae78e1b121cea102 on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:30:33.950146 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-ca43ebdd-60f5-42f8-8596-1f369151d583 tempest-TestShelveInstance-1915169858 tempest-TestShelveInstance-1915169858-project-admin] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 1094 time: 0.2132914 May 24 11:30:33.953610 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34856) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:30:33.958691 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.v2.base [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Request body: {'floatingip': {'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'port_id': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'fixed_ip_address': '172.20.0.11'}} {{(pid=91819) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 24 11:30:33.964954 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 24 11:30:33.969580 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Usage tracker for resource:floatingip and tenant:4ee563f50afe4088aebf60ed28c59ea7 is out of sync, need to count used quota {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 24 11:30:33.972537 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91819) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 24 11:30:33.978830 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.quota.driver [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91819) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 24 11:30:33.990006 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for floatingip, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:34.145591 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:34.309855 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_common [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Allocated IP 172.27.0.100 (858b8fad-a73f-4578-9079-6e731b9ff7e2/a677ede7-728d-486b-819a-5fc603fb3f9b/3551d891-792f-4dc5-8f46-a9b854d001c2) {{(pid=91819) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 24 11:30:34.466273 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for port, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:34.502907 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8779718821084', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:34.505042 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-a096cee9-3e12-4e4e-8a6d-2cbffc81bc7e service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:30:34.569960 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Network 858b8fad-a73f-4578-9079-6e731b9ff7e2 is already hosted by enough agents. {{(pid=91819) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 24 11:30:34.572788 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.base_scheduler [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Resources bound (agent type, host, resource id): {{(pid=91819) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 24 11:30:34.576804 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Received port after_create {{(pid=91819) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 24 11:30:34.611253 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event updated for resources: {'Port': ['ID=3551d891-792f-4dc5-8f46-a9b854d001c2,revision_number=1']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:30:34.617463 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:30:35.013119 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:35.096348 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:35.457580 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for port, precommit_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:35.462277 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-535155582'] for port, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:35.466166 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d3b82bbe-dcdf-486a-aa9c-9c516ed4f9dd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:30:35.591173 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event updated for resources: {'Port': ['ID=3551d891-792f-4dc5-8f46-a9b854d001c2,revision_number=2']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:30:35.599623 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.133s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:30:35.608989 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-686939'] for floatingip, after_update {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:35.789422 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Floating IP c6b3f275-b04b-4c9b-8d54-42d4849eb32a associated. External IP: 172.27.0.100, port: 66674db0-9a62-4491-be8b-38bf92ccfa58. May 24 11:30:36.220809 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:30:36.417631 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:30:36.426928 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c686be3b-d5b2-4402-bfe7-cab1887f6f68 None None] Sync routers for ids ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:30:36.436028 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c686be3b-d5b2-4402-bfe7-cab1887f6f68 None None] Routers {'2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:30:36.436668 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-c686be3b-d5b2-4402-bfe7-cab1887f6f68 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:30:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:30:36.444199 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:36.503354 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 997 time: 2.5486231 May 24 11:30:36.504019 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:30:36.554892 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'status': 'completed', 'code': 200} May 24 11:30:36.667677 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-c686be3b-d5b2-4402-bfe7-cab1887f6f68 None None] Set router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:30:36.670701 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d4f2cff-993d-4535-ae00-9cee301bd948 req-43f952f4-a872-428e-9327-2d0da4e3be3f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:36.673772 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d4f2cff-993d-4535-ae00-9cee301bd948 req-43f952f4-a872-428e-9327-2d0da4e3be3f service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=4ee563f50afe4088aebf60ed28c59ea7&device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 1099 time: 0.0964048 May 24 11:30:36.812659 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d4f2cff-993d-4535-ae00-9cee301bd948 req-925d1519-183d-4860-b5c7-7b81aaa04ddd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:36.814694 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d4f2cff-993d-4535-ae00-9cee301bd948 req-925d1519-183d-4860-b5c7-7b81aaa04ddd service neutron] 10.222.0.93 "GET /v2.0/networks?id=496a7264-ce0a-4dd1-b29d-67154eb68a39 HTTP/1.1" status: 200 len: 909 time: 0.1353614 May 24 11:30:37.056276 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d4f2cff-993d-4535-ae00-9cee301bd948 req-c7d652a0-422a-41be-906a-a5d3b17be7d2 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=66674db0-9a62-4491-be8b-38bf92ccfa58 HTTP/1.1" status: 200 len: 1000 time: 0.2352452 May 24 11:30:37.063117 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-c686be3b-d5b2-4402-bfe7-cab1887f6f68 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 4, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:51Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': 'c6b3f275-b04b-4c9b-8d54-42d4849eb32a', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'floating_ip_address': '172.27.0.100', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'port_id': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'fixed_ip_address': '172.20.0.11', 'status': 'DOWN', 'standard_attr_id': 687, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'mac_address': 'fa:16:3e:8b:76:f6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:30:34Z', 'updated_at': '2021-05-24T11:30:34Z', 'revision_number': 0, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'a61e4d23-b4eb-4110-a731-0d50771b3303', 'name': '', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'mac_address': 'fa:16:3e:f6:4c:6d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_interface', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '86fc06b2-ed33-40b0-849c-081575fab447', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:47Z', 'updated_at': '2021-05-24T11:29:55Z', 'revision_number': 5, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'subnets': [{'id': '86fc06b2-ed33-40b0-849c-081575fab447', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:30:37.067321 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c686be3b-d5b2-4402-bfe7-cab1887f6f68 None None] Checking router: 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:30:37.076022 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c686be3b-d5b2-4402-bfe7-cab1887f6f68 None None] Routers {'2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:30:37.076649 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-c686be3b-d5b2-4402-bfe7-cab1887f6f68 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:30:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:30:37.120008 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d4f2cff-993d-4535-ae00-9cee301bd948 req-fd0d6e4f-6993-45a7-96ae-37854a5cbe07 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:37.121478 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d4f2cff-993d-4535-ae00-9cee301bd948 req-fd0d6e4f-6993-45a7-96ae-37854a5cbe07 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=86fc06b2-ed33-40b0-849c-081575fab447 HTTP/1.1" status: 200 len: 847 time: 0.0593426 May 24 11:30:37.204847 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d4f2cff-993d-4535-ae00-9cee301bd948 req-cb93d752-0367-420e-ab6a-65695de69d86 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:30:37.206377 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d4f2cff-993d-4535-ae00-9cee301bd948 req-cb93d752-0367-420e-ab6a-65695de69d86 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=496a7264-ce0a-4dd1-b29d-67154eb68a39&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0788453 May 24 11:30:37.271118 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-c686be3b-d5b2-4402-bfe7-cab1887f6f68 None None] Set router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:30:37.344134 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d4f2cff-993d-4535-ae00-9cee301bd948 req-56894d3f-2af1-449b-a0d6-5e6f598dfda9 service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1308742 May 24 11:30:37.482353 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1d4f2cff-993d-4535-ae00-9cee301bd948 req-3548018b-3570-4628-b772-37309fc04ba5 service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1320784 May 24 11:30:37.648361 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-c686be3b-d5b2-4402-bfe7-cab1887f6f68 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 4, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:51Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': 'c6b3f275-b04b-4c9b-8d54-42d4849eb32a', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'floating_ip_address': '172.27.0.100', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'port_id': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'fixed_ip_address': '172.20.0.11', 'status': 'DOWN', 'standard_attr_id': 687, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'mac_address': 'fa:16:3e:8b:76:f6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:30:34Z', 'updated_at': '2021-05-24T11:30:34Z', 'revision_number': 0, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'a61e4d23-b4eb-4110-a731-0d50771b3303', 'name': '', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'mac_address': 'fa:16:3e:f6:4c:6d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_interface', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '86fc06b2-ed33-40b0-849c-081575fab447', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:47Z', 'updated_at': '2021-05-24T11:29:55Z', 'revision_number': 5, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'subnets': [{'id': '86fc06b2-ed33-40b0-849c-081575fab447', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:30:37.652558 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c686be3b-d5b2-4402-bfe7-cab1887f6f68 None None] The sync data for ids ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'] in dv-cinder-784092-9-iscsi is: [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 4, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:51Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_floatingips': [{'id': 'c6b3f275-b04b-4c9b-8d54-42d4849eb32a', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'floating_ip_address': '172.27.0.100', 'floating_network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'router_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'port_id': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'fixed_ip_address': '172.20.0.11', 'status': 'DOWN', 'standard_attr_id': 687, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'mac_address': 'fa:16:3e:8b:76:f6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-24T11:30:34Z', 'updated_at': '2021-05-24T11:30:34Z', 'revision_number': 0, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'a61e4d23-b4eb-4110-a731-0d50771b3303', 'name': '', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'mac_address': 'fa:16:3e:f6:4c:6d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_interface', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '86fc06b2-ed33-40b0-849c-081575fab447', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:47Z', 'updated_at': '2021-05-24T11:29:55Z', 'revision_number': 5, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'subnets': [{'id': '86fc06b2-ed33-40b0-849c-081575fab447', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:30:37.852449 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7d7893b2-e7f3-4ab3-a597-8406dd5cb581 None None] New status for floating IP c6b3f275-b04b-4c9b-8d54-42d4849eb32a: ACTIVE {{(pid=91822) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:30:52.142666 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:53.459918 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-0b1c801a-9635-4372-9a80-713bad9a420c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:55.790174 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:55.898096 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a5aaddbe-3980-4636-9f1e-87820a5d8d6a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:57.036835 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:30:58.975517 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:31:07.306048 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:31:22.140864 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:31:23.462730 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-fe749fb5-96b6-4965-9ffc-5e3ee44a5790 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:31:25.794053 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:31:25.900377 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-b845f493-d27d-4366-be98-e42620aa3fea None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:31:27.033708 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:31:28.904783 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932310d9-52f3-4002-91da-e1aefddf5178 req-3aee65ff-dc33-4083-a70d-a2f66e171d43 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0842559 May 24 11:31:28.989756 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:31:29.075741 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-932310d9-52f3-4002-91da-e1aefddf5178 req-a1321842-0c7b-4edd-a506-8dc0efb7b873 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:31:29.077325 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932310d9-52f3-4002-91da-e1aefddf5178 req-a1321842-0c7b-4edd-a506-8dc0efb7b873 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=4ee563f50afe4088aebf60ed28c59ea7&device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 1099 time: 0.0746226 May 24 11:31:29.236912 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-932310d9-52f3-4002-91da-e1aefddf5178 req-3e8e3a17-6505-4b39-a937-1541cede116e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:31:29.238513 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932310d9-52f3-4002-91da-e1aefddf5178 req-3e8e3a17-6505-4b39-a937-1541cede116e service neutron] 10.222.0.93 "GET /v2.0/networks?id=496a7264-ce0a-4dd1-b29d-67154eb68a39 HTTP/1.1" status: 200 len: 909 time: 0.1425383 May 24 11:31:29.354274 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932310d9-52f3-4002-91da-e1aefddf5178 req-8bc5dab7-3f5c-470f-91b4-e1af7ef64490 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=66674db0-9a62-4491-be8b-38bf92ccfa58 HTTP/1.1" status: 200 len: 1002 time: 0.1093192 May 24 11:31:29.419467 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-932310d9-52f3-4002-91da-e1aefddf5178 req-2b27d39d-71b4-4877-a2f2-37979f9e5351 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:31:29.420955 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932310d9-52f3-4002-91da-e1aefddf5178 req-2b27d39d-71b4-4877-a2f2-37979f9e5351 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=86fc06b2-ed33-40b0-849c-081575fab447 HTTP/1.1" status: 200 len: 847 time: 0.0605991 May 24 11:31:29.503650 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-932310d9-52f3-4002-91da-e1aefddf5178 req-159c5914-c48c-4615-9a01-8395d286dd2d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:31:29.504919 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932310d9-52f3-4002-91da-e1aefddf5178 req-159c5914-c48c-4615-9a01-8395d286dd2d service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=496a7264-ce0a-4dd1-b29d-67154eb68a39&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0782421 May 24 11:31:29.654959 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932310d9-52f3-4002-91da-e1aefddf5178 req-0ce47b16-55eb-4d91-b2bc-f76817ca165f service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1426618 May 24 11:31:29.809334 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-932310d9-52f3-4002-91da-e1aefddf5178 req-e68403e2-6dde-4673-a81e-d5f18d066a15 service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1483328 May 24 11:31:44.306883 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:31:45.075503 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [req-c8f9817e-106b-4992-93fc-22bd958365aa req-7e0d6c9b-d24c-4159-af93-40bd332e64a5 service neutron] (91819) timed out ('10.222.0.93', 65201) {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 24 11:31:52.159729 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:31:53.468108 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3a9dda92-2643-4474-8656-a3dae9106111 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:31:55.799139 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:31:55.907017 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-0e6190da-cfcf-45ab-b507-b7232b85267d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:31:57.037311 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:31:58.980537 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:21.305641 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:32:22.159080 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:23.470687 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aaa70bc6-a510-4b59-b0ea-63d0e73d03f5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:25.803498 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:25.907694 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-35bd18b4-7a18-4083-a5db-4cdb965d813c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:26.216498 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e963a96-49c4-4a04-88c2-f090ed94d25f req-eb5f7c82-b5ae-459b-ad6c-1d6e221641cd tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:26.218000 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0e963a96-49c4-4a04-88c2-f090ed94d25f req-eb5f7c82-b5ae-459b-ad6c-1d6e221641cd tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.1365037 May 24 11:32:26.266725 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-0e963a96-49c4-4a04-88c2-f090ed94d25f req-81fe3adb-73fd-43d9-8cea-ccbdd5efde0a tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0403256 May 24 11:32:27.037005 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:27.512392 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d356d82-bdad-4901-bc01-d30587a47ece req-c8f8575c-5469-42b6-96ab-2541a6602df9 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:27.513938 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7d356d82-bdad-4901-bc01-d30587a47ece req-c8f8575c-5469-42b6-96ab-2541a6602df9 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.1390979 May 24 11:32:27.569768 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-7d356d82-bdad-4901-bc01-d30587a47ece req-8b97ef48-14eb-46a2-908e-ba5fe4977c60 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0496869 May 24 11:32:28.767418 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ecf5dd2e-9663-4caf-b5a0-ef53a360c7c1 req-afcacbb9-4859-44da-a98a-2e06b6563f58 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:28.768903 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ecf5dd2e-9663-4caf-b5a0-ef53a360c7c1 req-afcacbb9-4859-44da-a98a-2e06b6563f58 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0810065 May 24 11:32:28.816371 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-ecf5dd2e-9663-4caf-b5a0-ef53a360c7c1 req-6b3fff16-9cc1-4164-a07e-f1f2f3345861 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0403519 May 24 11:32:28.983435 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:30.019718 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5899a9cb-aa0b-4071-9ccd-c9b229ef0318 req-9c981005-894a-40c0-b555-dde5b732d602 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:30.021736 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5899a9cb-aa0b-4071-9ccd-c9b229ef0318 req-9c981005-894a-40c0-b555-dde5b732d602 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0891197 May 24 11:32:30.072415 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5899a9cb-aa0b-4071-9ccd-c9b229ef0318 req-a1ce8376-9ade-45b3-ae7f-bf928de454db tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0432224 May 24 11:32:31.042499 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bd657ca8-502a-4421-b182-f329e786e4fb req-b37eb933-a977-49c7-93ef-b428ce0b4622 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:31.048399 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bd657ca8-502a-4421-b182-f329e786e4fb req-b37eb933-a977-49c7-93ef-b428ce0b4622 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=4ee563f50afe4088aebf60ed28c59ea7&device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 1099 time: 0.1413069 May 24 11:32:31.310140 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e31844ac-5926-4bc1-ab6b-ab88ec44a431 req-37b635d5-0d6f-42d5-9224-29b203e08ed7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:31.311961 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e31844ac-5926-4bc1-ab6b-ab88ec44a431 req-37b635d5-0d6f-42d5-9224-29b203e08ed7 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.1089606 May 24 11:32:31.360307 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bd657ca8-502a-4421-b182-f329e786e4fb req-5505c71c-9a3d-486c-86a2-0447c0605561 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:31.361759 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bd657ca8-502a-4421-b182-f329e786e4fb req-5505c71c-9a3d-486c-86a2-0447c0605561 service neutron] 10.222.0.93 "GET /v2.0/networks?id=496a7264-ce0a-4dd1-b29d-67154eb68a39 HTTP/1.1" status: 200 len: 909 time: 0.2921844 May 24 11:32:31.366904 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-e31844ac-5926-4bc1-ab6b-ab88ec44a431 req-5337157e-b8d5-4e2e-8901-8b863195d111 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0478916 May 24 11:32:31.475641 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bd657ca8-502a-4421-b182-f329e786e4fb req-5edc679b-95fc-4abb-beba-5e6395a45ba1 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=66674db0-9a62-4491-be8b-38bf92ccfa58 HTTP/1.1" status: 200 len: 1002 time: 0.1056023 May 24 11:32:31.540088 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bd657ca8-502a-4421-b182-f329e786e4fb req-6730d7ff-dfc8-4610-ab12-8a06cb2d17f6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:31.541562 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bd657ca8-502a-4421-b182-f329e786e4fb req-6730d7ff-dfc8-4610-ab12-8a06cb2d17f6 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=86fc06b2-ed33-40b0-849c-081575fab447 HTTP/1.1" status: 200 len: 847 time: 0.0599322 May 24 11:32:31.618565 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bd657ca8-502a-4421-b182-f329e786e4fb req-b421e82a-45b5-4f24-ac0f-030f0d479caa service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:31.619957 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bd657ca8-502a-4421-b182-f329e786e4fb req-b421e82a-45b5-4f24-ac0f-030f0d479caa service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=496a7264-ce0a-4dd1-b29d-67154eb68a39&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0722089 May 24 11:32:31.755736 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bd657ca8-502a-4421-b182-f329e786e4fb req-540baaa8-e214-44ee-b630-c6a051cd3f8b service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1282451 May 24 11:32:31.891355 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-bd657ca8-502a-4421-b182-f329e786e4fb req-b94aa73c-b196-4613-8010-dd4c6a543564 service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1288357 May 24 11:32:32.575282 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d5848aaf-3011-4dc9-bace-3d715e2d128f req-1de60dd4-282d-4677-9b60-e92e2502423a tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:32.576808 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d5848aaf-3011-4dc9-bace-3d715e2d128f req-1de60dd4-282d-4677-9b60-e92e2502423a tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0894494 May 24 11:32:32.630224 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d5848aaf-3011-4dc9-bace-3d715e2d128f req-850f0a8c-3245-4268-b98a-323b65144957 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0435655 May 24 11:32:33.827543 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9a5f3a2a-f2ed-4cf5-a088-674260a1c706 req-9ced1148-294a-47da-87a8-8a472530acbb tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:33.828922 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9a5f3a2a-f2ed-4cf5-a088-674260a1c706 req-9ced1148-294a-47da-87a8-8a472530acbb tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0808802 May 24 11:32:33.878369 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9a5f3a2a-f2ed-4cf5-a088-674260a1c706 req-0926d632-358b-4f29-90bc-ed3dec7de177 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0436130 May 24 11:32:35.068387 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1651385-753a-44ab-a101-608580b22cd7 req-ba785efb-8db3-454d-9c19-7915cb034dc9 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:35.069938 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f1651385-753a-44ab-a101-608580b22cd7 req-ba785efb-8db3-454d-9c19-7915cb034dc9 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0817800 May 24 11:32:35.119924 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-f1651385-753a-44ab-a101-608580b22cd7 req-d9941d56-3e4e-4abd-a3c9-d081e465bc68 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0436778 May 24 11:32:36.280227 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc725385-7056-48bc-a9a4-1dda6b1c98d1 req-e6f42dbc-7075-4acc-98c6-66bd76ffe32f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:36.281747 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dc725385-7056-48bc-a9a4-1dda6b1c98d1 req-e6f42dbc-7075-4acc-98c6-66bd76ffe32f service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=4ee563f50afe4088aebf60ed28c59ea7&device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 1099 time: 0.0782974 May 24 11:32:36.332070 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8481f954-48ed-47e7-9f3b-8f15f0af0a70 req-6a9f0b68-0947-4e41-b7b0-5cfd0708f1fb tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:36.333617 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8481f954-48ed-47e7-9f3b-8f15f0af0a70 req-6a9f0b68-0947-4e41-b7b0-5cfd0708f1fb tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0892873 May 24 11:32:36.383926 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-8481f954-48ed-47e7-9f3b-8f15f0af0a70 req-faf73ab2-a9d3-4dcc-a76e-cf2bac46cf69 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0431421 May 24 11:32:36.422024 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc725385-7056-48bc-a9a4-1dda6b1c98d1 req-260ad850-62b0-4a82-a5f8-63a33a59bf80 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:36.423395 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dc725385-7056-48bc-a9a4-1dda6b1c98d1 req-260ad850-62b0-4a82-a5f8-63a33a59bf80 service neutron] 10.222.0.93 "GET /v2.0/networks?id=496a7264-ce0a-4dd1-b29d-67154eb68a39 HTTP/1.1" status: 200 len: 909 time: 0.1353953 May 24 11:32:36.532528 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dc725385-7056-48bc-a9a4-1dda6b1c98d1 req-9c946c55-8c96-443f-a25e-542fde7403d6 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=66674db0-9a62-4491-be8b-38bf92ccfa58 HTTP/1.1" status: 200 len: 1002 time: 0.1028411 May 24 11:32:36.597897 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc725385-7056-48bc-a9a4-1dda6b1c98d1 req-e8b88bf6-0467-4579-8fa4-031dfde59a20 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:36.599331 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dc725385-7056-48bc-a9a4-1dda6b1c98d1 req-e8b88bf6-0467-4579-8fa4-031dfde59a20 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=86fc06b2-ed33-40b0-849c-081575fab447 HTTP/1.1" status: 200 len: 847 time: 0.0612297 May 24 11:32:36.683561 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc725385-7056-48bc-a9a4-1dda6b1c98d1 req-e0711991-79d9-4066-8fcb-9b29ddb0d99d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:36.685143 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dc725385-7056-48bc-a9a4-1dda6b1c98d1 req-e0711991-79d9-4066-8fcb-9b29ddb0d99d service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=496a7264-ce0a-4dd1-b29d-67154eb68a39&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0796361 May 24 11:32:36.863603 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dc725385-7056-48bc-a9a4-1dda6b1c98d1 req-bce0fe42-4746-4887-88f7-d2ce8a79525c service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1722894 May 24 11:32:37.007331 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-dc725385-7056-48bc-a9a4-1dda6b1c98d1 req-80c4f80e-09d8-4d84-83f8-7d495c518498 service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1357956 May 24 11:32:37.572329 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32684a67-56ab-4895-81e4-89a11d75ea4c req-7b5418c2-bf0a-41e9-9ee4-76948812cdcb tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:37.574358 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-32684a67-56ab-4895-81e4-89a11d75ea4c req-7b5418c2-bf0a-41e9-9ee4-76948812cdcb tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0825694 May 24 11:32:37.619275 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-32684a67-56ab-4895-81e4-89a11d75ea4c req-f1886aff-1b51-47e2-94af-4844925876af tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0392070 May 24 11:32:38.807462 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d3d92e8d-f98b-4e20-9ea1-211a0aabf379 req-1f2ba39b-4a3d-452a-95c0-6c84bcdc76c4 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:38.808890 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d3d92e8d-f98b-4e20-9ea1-211a0aabf379 req-1f2ba39b-4a3d-452a-95c0-6c84bcdc76c4 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0841863 May 24 11:32:38.862236 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-d3d92e8d-f98b-4e20-9ea1-211a0aabf379 req-24290421-d268-4cf4-b090-9a1b9d8e7f74 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0472059 May 24 11:32:40.173291 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-855afb23-48b1-47f4-95db-505a81cd71ac req-2b7c836a-f9dc-4c9a-a378-511aad884378 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:40.174720 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-855afb23-48b1-47f4-95db-505a81cd71ac req-2b7c836a-f9dc-4c9a-a378-511aad884378 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.1979029 May 24 11:32:40.226394 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-855afb23-48b1-47f4-95db-505a81cd71ac req-af89ff79-274d-4e0b-8873-d81781216976 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0433724 May 24 11:32:40.562778 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-9b885123-2ab6-4c1b-a964-242744237fbf req-5aa651fd-35b3-4280-ac4c-802afb2339ad service neutron] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.1187160 May 24 11:32:40.584942 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1315050e-0415-471a-a596-34afdd4da42c req-678269ee-1560-44be-a731-078dd0b11194 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:40.586601 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1315050e-0415-471a-a596-34afdd4da42c req-678269ee-1560-44be-a731-078dd0b11194 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0842147 May 24 11:32:40.635960 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-1315050e-0415-471a-a596-34afdd4da42c req-78d3f93d-f096-4079-aa02-058a63a167ff tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0433357 May 24 11:32:41.434436 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 66674db0-9a62-4491-be8b-38bf92ccfa58 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:32:41.547345 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:41.578207 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:32:41.590191 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 66674db0-9a62-4491-be8b-38bf92ccfa58, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=66674db0-9a62-4491-be8b-38bf92ccfa58,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:32:41.595386 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:41.598064 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:32:41.695803 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=66674db0-9a62-4491-be8b-38bf92ccfa58,revision_number=6']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:32:41.710464 dv-cinder-784092-9-iscsi neutron-server[91821]: WARNING neutron.notifiers.nova [-] Nova event: {'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'name': 'network-vif-unplugged', 'status': 'failed', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'code': 422} returned with failed status May 24 11:32:41.712900 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:32:41.713531 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:32:41.714126 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:32:41.714696 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:32:41.715252 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:32:41.715818 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:32:41.716380 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:32:41.716948 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:32:41.717541 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:32:41.718103 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:32:41.718661 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:32:41.719220 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:32:41.725733 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.128s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:32:41.870883 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6fa6678f-3231-4c49-b357-826ecd51e883 req-da8261e1-c91b-4cf7-a87e-dc5cb361501c tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:41.872493 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6fa6678f-3231-4c49-b357-826ecd51e883 req-da8261e1-c91b-4cf7-a87e-dc5cb361501c tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0852797 May 24 11:32:41.918254 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6fa6678f-3231-4c49-b357-826ecd51e883 req-4be1329d-7544-4734-880b-58e5e570220c tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0400574 May 24 11:32:43.243423 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6abc3de-1d3b-419e-911b-69f6113db40c req-d5fc8a90-4a8d-4aba-bb5f-76fb44f0eaa1 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:43.244908 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b6abc3de-1d3b-419e-911b-69f6113db40c req-d5fc8a90-4a8d-4aba-bb5f-76fb44f0eaa1 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.2059126 May 24 11:32:43.291938 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-b6abc3de-1d3b-419e-911b-69f6113db40c req-7069c693-d1a6-43a7-bb30-f1f0871eda32 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0404661 May 24 11:32:44.487763 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-348b462b-9ca0-477b-baa7-900a6913a448 req-3b8a3662-0f71-4626-9920-0708cd1c41b8 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:44.489269 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-348b462b-9ca0-477b-baa7-900a6913a448 req-3b8a3662-0f71-4626-9920-0708cd1c41b8 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0926926 May 24 11:32:44.536276 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-348b462b-9ca0-477b-baa7-900a6913a448 req-da9ca7d4-0b1c-4519-b9e1-986ebdf5475d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0413468 May 24 11:32:45.737364 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55ed77e6-216c-4574-8bc8-7a41c6a87f4d req-f3c7d004-d2f7-4258-95f5-a44f0fc627c6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:45.738719 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-55ed77e6-216c-4574-8bc8-7a41c6a87f4d req-f3c7d004-d2f7-4258-95f5-a44f0fc627c6 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0931609 May 24 11:32:45.788052 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-55ed77e6-216c-4574-8bc8-7a41c6a87f4d req-d3757a80-2760-4101-ba1b-c606721829c3 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0425522 May 24 11:32:46.505893 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b885123-2ab6-4c1b-a964-242744237fbf req-b67553a0-d6ad-4cc7-91c6-fec2081bc1dd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:46.507350 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b885123-2ab6-4c1b-a964-242744237fbf req-b67553a0-d6ad-4cc7-91c6-fec2081bc1dd service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7&tenant_id=4ee563f50afe4088aebf60ed28c59ea7 HTTP/1.1" status: 200 len: 1097 time: 0.0827203 May 24 11:32:46.610777 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b885123-2ab6-4c1b-a964-242744237fbf req-32055acd-416e-454f-8e4e-80c6a8d5bcf8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:46.612224 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b885123-2ab6-4c1b-a964-242744237fbf req-32055acd-416e-454f-8e4e-80c6a8d5bcf8 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=4ee563f50afe4088aebf60ed28c59ea7&device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 1097 time: 0.0772672 May 24 11:32:46.746114 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b885123-2ab6-4c1b-a964-242744237fbf req-cb15e741-c608-45db-be79-47a76805ee55 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:46.747545 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b885123-2ab6-4c1b-a964-242744237fbf req-cb15e741-c608-45db-be79-47a76805ee55 service neutron] 10.222.0.93 "GET /v2.0/networks?id=496a7264-ce0a-4dd1-b29d-67154eb68a39 HTTP/1.1" status: 200 len: 909 time: 0.1296833 May 24 11:32:46.852024 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b885123-2ab6-4c1b-a964-242744237fbf req-1226c764-177a-41a9-89eb-072fa591998d service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=66674db0-9a62-4491-be8b-38bf92ccfa58 HTTP/1.1" status: 200 len: 1000 time: 0.0982559 May 24 11:32:46.962141 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b885123-2ab6-4c1b-a964-242744237fbf req-f680db9c-98c2-427e-ad52-4ad053a38285 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:46.963596 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b885123-2ab6-4c1b-a964-242744237fbf req-f680db9c-98c2-427e-ad52-4ad053a38285 service neutron] 10.222.0.93 "GET /v2.0/subnets?id=86fc06b2-ed33-40b0-849c-081575fab447 HTTP/1.1" status: 200 len: 847 time: 0.1059012 May 24 11:32:47.047547 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10538e77-fc93-4d9a-b438-1da62970ac5e req-e9976135-5d44-4f47-be74-41f4ed0798e8 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:47.049071 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-10538e77-fc93-4d9a-b438-1da62970ac5e req-e9976135-5d44-4f47-be74-41f4ed0798e8 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.1533947 May 24 11:32:47.088168 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b885123-2ab6-4c1b-a964-242744237fbf req-7de700e9-de07-471a-85b1-1ac73276f4a0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:47.090059 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b885123-2ab6-4c1b-a964-242744237fbf req-7de700e9-de07-471a-85b1-1ac73276f4a0 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=496a7264-ce0a-4dd1-b29d-67154eb68a39&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1200218 May 24 11:32:47.119493 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-10538e77-fc93-4d9a-b438-1da62970ac5e req-6993ac81-ef88-4ff1-a6f5-ddc849443ed1 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0643740 May 24 11:32:47.230771 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b885123-2ab6-4c1b-a964-242744237fbf req-f216b60b-de68-4d92-9e72-2cbcad25d9a9 service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1327415 May 24 11:32:47.363774 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-9b885123-2ab6-4c1b-a964-242744237fbf req-3db1c9e6-ce74-41b6-aafd-beab41698dee service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1266158 May 24 11:32:48.313312 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3c16dce8-1fef-403a-a39b-eaf28b332361 req-7b3af787-0b59-48f5-ba9a-8324382f59ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:48.314974 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3c16dce8-1fef-403a-a39b-eaf28b332361 req-7b3af787-0b59-48f5-ba9a-8324382f59ef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0891645 May 24 11:32:48.361373 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-3c16dce8-1fef-403a-a39b-eaf28b332361 req-80c68da6-87d5-4ce5-ba57-8d5eb82bfb8a tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0402734 May 24 11:32:49.556964 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5d69b3a1-3366-474f-b385-ab2d9629fa21 req-d16cc1ea-0283-47a2-9f47-1d3587d2a3e1 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:49.558526 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5d69b3a1-3366-474f-b385-ab2d9629fa21 req-d16cc1ea-0283-47a2-9f47-1d3587d2a3e1 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0873523 May 24 11:32:49.610436 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5d69b3a1-3366-474f-b385-ab2d9629fa21 req-5b9de6ea-c1eb-4eca-bfe2-a75688a42e25 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0459926 May 24 11:32:50.795692 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5662add7-75df-456e-af36-da32a3431e73 req-787e0f70-6bd0-445c-a721-1f7dbf2ab55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:50.797194 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5662add7-75df-456e-af36-da32a3431e73 req-787e0f70-6bd0-445c-a721-1f7dbf2ab55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0823178 May 24 11:32:50.848953 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-5662add7-75df-456e-af36-da32a3431e73 req-a9a0d54e-3df3-48af-919e-d6b372f2cb44 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0453868 May 24 11:32:52.033239 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-714ef6c9-53b5-4fba-9070-dff4d8da6c17 req-98363cc4-b89f-474d-9588-9dd64ec9354b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:52.034703 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-714ef6c9-53b5-4fba-9070-dff4d8da6c17 req-98363cc4-b89f-474d-9588-9dd64ec9354b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0814691 May 24 11:32:52.080181 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-714ef6c9-53b5-4fba-9070-dff4d8da6c17 req-8eab73e7-cdba-4597-a531-781428908781 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0392828 May 24 11:32:52.197128 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:53.279109 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f4e1764c-001b-477d-a392-26270c1874a2 req-9a3bdbef-6f8e-4969-868a-f7f8beebc1de tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:53.280693 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f4e1764c-001b-477d-a392-26270c1874a2 req-9a3bdbef-6f8e-4969-868a-f7f8beebc1de tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0830777 May 24 11:32:53.327610 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f4e1764c-001b-477d-a392-26270c1874a2 req-2b116fbd-aef4-4774-b382-9ea6ab564e0e tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0411646 May 24 11:32:53.476756 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-d489ddb7-c270-4f22-9526-d86ea8e4333f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:54.521553 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-13d2fbc4-8728-4420-a857-c985fc9b4a3c req-14cf01b8-63e9-4986-b92e-5e93e3afc66e tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:54.523035 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-13d2fbc4-8728-4420-a857-c985fc9b4a3c req-14cf01b8-63e9-4986-b92e-5e93e3afc66e tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.0876517 May 24 11:32:54.572535 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-13d2fbc4-8728-4420-a857-c985fc9b4a3c req-cc0c2148-2a29-419a-8f26-62480396c5bc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0429695 May 24 11:32:55.512276 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 66674db0-9a62-4491-be8b-38bf92ccfa58 details requested by agent hyperv_hv-cinder-78409 with host hv-cinder-78409 {{(pid=91821) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 24 11:32:55.663377 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 66674db0-9a62-4491-be8b-38bf92ccfa58, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=66674db0-9a62-4491-be8b-38bf92ccfa58,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:32:55.678169 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Returning: {'device': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'port_id': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'mac_address': 'fa:16:3e:8b:76:f6', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 540, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '86fc06b2-ed33-40b0-849c-081575fab447', 'ip_address': '172.20.0.11'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91821) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 24 11:32:55.734214 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:55.751946 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 66674db0-9a62-4491-be8b-38bf92ccfa58 {{(pid=91821) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 24 11:32:55.767621 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 66674db0-9a62-4491-be8b-38bf92ccfa58, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=66674db0-9a62-4491-be8b-38bf92ccfa58,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:32:55.776801 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:55.779208 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-46d24114-42cf-4dc3-ad05-93eb4a0ec306 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:32:55.793719 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-067edc98-0d09-42f7-98b1-ccc36bab9b6b req-cf6ae251-e3fe-436c-a707-4de8fa9f4e4d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:55.795220 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-067edc98-0d09-42f7-98b1-ccc36bab9b6b req-cf6ae251-e3fe-436c-a707-4de8fa9f4e4d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 944 time: 0.1031775 May 24 11:32:55.844741 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-067edc98-0d09-42f7-98b1-ccc36bab9b6b req-40901740-6964-4e8c-94a4-7bfe998c3c69 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0425854 May 24 11:32:55.849513 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:55.866891 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=66674db0-9a62-4491-be8b-38bf92ccfa58,revision_number=7']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:32:55.870074 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:32:55.899675 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-acef048d-8f31-4124-b099-79e4922cd0f7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:56.002549 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] get_ports_and_sgs() called for port_ids ['66674db0-9a62-4491-be8b-38bf92ccfa58'] {{(pid=91821) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 24 11:32:57.040669 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:57.064627 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aef11bd1-b523-4172-9f98-9cc4a798762a req-f23972df-81cc-474a-9eb3-f7b974261ae0 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:57.066209 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-aef11bd1-b523-4172-9f98-9cc4a798762a req-f23972df-81cc-474a-9eb3-f7b974261ae0 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 945 time: 0.0875251 May 24 11:32:57.114330 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-aef11bd1-b523-4172-9f98-9cc4a798762a req-22d75a78-0d33-4e95-9cf6-53327a890b17 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0413756 May 24 11:32:57.541920 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 66674db0-9a62-4491-be8b-38bf92ccfa58 up at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 24 11:32:57.618307 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.provisioning_blocks [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Provisioning complete for port 66674db0-9a62-4491-be8b-38bf92ccfa58 triggered by entity L2. {{(pid=91821) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 24 11:32:57.618307 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1889181'] for port, provisioning_complete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:57.740065 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-1953195'] for port, before_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:57.770295 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58'}] {{(pid=91821) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:32:57.781183 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] For port 66674db0-9a62-4491-be8b-38bf92ccfa58, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=66674db0-9a62-4491-be8b-38bf92ccfa58,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:32:57.785903 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8779719271828', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8779718821102', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8779718825247'] for port, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:57.787847 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:32:57.823340 dv-cinder-784092-9-iscsi neutron-server[91821]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'code': 200} May 24 11:32:57.857035 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Pushing event updated for resources: {'Port': ['ID=66674db0-9a62-4491-be8b-38bf92ccfa58,revision_number=8']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:32:57.860226 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:32:58.206102 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2a86307-6208-4560-af6b-9c8d155ccb3e req-10f2320e-dde0-4e93-a752-0756d79525bb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:58.207622 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d2a86307-6208-4560-af6b-9c8d155ccb3e req-10f2320e-dde0-4e93-a752-0756d79525bb service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 1099 time: 0.1866214 May 24 11:32:58.251187 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d2a86307-6208-4560-af6b-9c8d155ccb3e req-afa1b96d-2a86-4363-9839-5cabbf223d03 service neutron] 10.222.0.93 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0370522 May 24 11:32:58.308959 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:32:58.329356 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b70bd201-5221-4688-ae1b-f9e77dc2bd85 req-af9a24b1-a691-4ca3-9e96-d6da5e0985f9 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:58.330833 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b70bd201-5221-4688-ae1b-f9e77dc2bd85 req-af9a24b1-a691-4ca3-9e96-d6da5e0985f9 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0841157 May 24 11:32:58.392252 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b70bd201-5221-4688-ae1b-f9e77dc2bd85 req-267f80e9-48cc-4cee-a232-1e1864b0928e tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0557384 May 24 11:32:58.993322 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:32:59.583411 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ba1ba12-90ba-4cce-bcdc-789a6ba8d200 req-f2349848-ba0b-4449-9dca-183e46c53cef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:32:59.584909 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4ba1ba12-90ba-4cce-bcdc-789a6ba8d200 req-f2349848-ba0b-4449-9dca-183e46c53cef tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0830522 May 24 11:32:59.631708 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-4ba1ba12-90ba-4cce-bcdc-789a6ba8d200 req-b5ecc944-0d90-4183-9d29-ef4aecb9e287 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0399327 May 24 11:33:22.152888 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:33:23.474405 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-5c2e158d-f64d-40db-8219-f030901f4dcc None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:33:25.805504 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:33:25.902647 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-51c4027d-1d45-4627-9579-52e76b07829f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:33:27.044230 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:33:28.988016 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:33:31.903955 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-283bf7db-f3be-4fab-a507-932cb3c8c1dc req-5b79d994-870a-49c6-96ec-436fe4b2dd46 service neutron] 10.222.0.93 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0737655 May 24 11:33:32.044953 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-283bf7db-f3be-4fab-a507-932cb3c8c1dc req-8b497d20-438b-421d-8aa3-84509d768c6c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:33:32.046470 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-283bf7db-f3be-4fab-a507-932cb3c8c1dc req-8b497d20-438b-421d-8aa3-84509d768c6c service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=4ee563f50afe4088aebf60ed28c59ea7&device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 1099 time: 0.0759888 May 24 11:33:32.197259 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-283bf7db-f3be-4fab-a507-932cb3c8c1dc req-25cffbb8-dbfa-4d5c-aeab-27b3004de082 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:33:32.198613 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-283bf7db-f3be-4fab-a507-932cb3c8c1dc req-25cffbb8-dbfa-4d5c-aeab-27b3004de082 service neutron] 10.222.0.93 "GET /v2.0/networks?id=496a7264-ce0a-4dd1-b29d-67154eb68a39 HTTP/1.1" status: 200 len: 909 time: 0.1301382 May 24 11:33:32.304973 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-283bf7db-f3be-4fab-a507-932cb3c8c1dc req-43ac6cad-fdb1-4a28-a7f3-586d6a1cc7b2 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=66674db0-9a62-4491-be8b-38bf92ccfa58 HTTP/1.1" status: 200 len: 1002 time: 0.0999341 May 24 11:33:32.372266 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-283bf7db-f3be-4fab-a507-932cb3c8c1dc req-cc596377-73ff-4b28-ae44-a796071a6baa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:33:32.373734 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-283bf7db-f3be-4fab-a507-932cb3c8c1dc req-cc596377-73ff-4b28-ae44-a796071a6baa service neutron] 10.222.0.93 "GET /v2.0/subnets?id=86fc06b2-ed33-40b0-849c-081575fab447 HTTP/1.1" status: 200 len: 847 time: 0.0624719 May 24 11:33:32.452628 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-283bf7db-f3be-4fab-a507-932cb3c8c1dc req-dd8838ff-f1bd-40a3-833c-b6706cdb4d57 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:33:32.454033 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-283bf7db-f3be-4fab-a507-932cb3c8c1dc req-dd8838ff-f1bd-40a3-833c-b6706cdb4d57 service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=496a7264-ce0a-4dd1-b29d-67154eb68a39&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0731053 May 24 11:33:32.594066 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-283bf7db-f3be-4fab-a507-932cb3c8c1dc req-407e545d-d42e-41bd-ad8b-9ccd608ec043 service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1333144 May 24 11:33:32.752620 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-283bf7db-f3be-4fab-a507-932cb3c8c1dc req-3d57b9d3-8224-48fc-9444-1005df729e2f service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1506088 May 24 11:33:35.308511 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:33:52.165068 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:33:53.470102 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-897c18b0-6152-493e-8421-f9e823db8b09 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:33:55.812210 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:33:55.911914 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-4ec427a4-9ab8-437b-ac99-ff1abd605877 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:33:57.046311 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:33:58.993119 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:11.806791 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 34994) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:12.074714 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Deleting port 3551d891-792f-4dc5-8f46-a9b854d001c2 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:34:12.075062 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:12.204057 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:12.204424 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:12.309570 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:34:12.330287 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Calling delete_port for 3551d891-792f-4dc5-8f46-a9b854d001c2 owned by network:floatingip {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:34:12.466708 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:12.470960 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-775f0bf1-881d-470b-a82e-ab6f59a63670 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:12.515610 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event deleted for resources: {'Port': ['ID=3551d891-792f-4dc5-8f46-a9b854d001c2,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:12.563307 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:12.563623 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:12.563992 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:12.564357 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:12.564711 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:12.565062 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:12.565441 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:12.565798 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:12.566149 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:12.566499 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:12.566847 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:12.567193 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91819) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:12.571178 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:12.738040 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for floatingip, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:12.738374 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.db.l3_db [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Floating IP c6b3f275-b04b-4c9b-8d54-42d4849eb32a disassociated (deleted). External IP: 172.27.0.100, port: 66674db0-9a62-4491-be8b-38bf92ccfa58. May 24 11:34:13.095231 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:34:13.277917 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:34:13.288120 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6a9e2484-5a5b-40e9-8453-0e47fe060eff None None] Sync routers for ids ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:34:13.298912 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for floatingip, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:13.301069 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6a9e2484-5a5b-40e9-8453-0e47fe060eff None None] Routers {'2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:34:13.301784 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-6a9e2484-5a5b-40e9-8453-0e47fe060eff None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:33:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:13.355714 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-7c12e2dc-e9d0-4d22-afa1-501973a80b0d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "DELETE /v2.0/floatingips/c6b3f275-b04b-4c9b-8d54-42d4849eb32a HTTP/1.1" status: 204 len: 168 time: 1.5478289 May 24 11:34:13.356370 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:34:13.482482 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'status': 'completed', 'code': 200} May 24 11:34:13.489040 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-6a9e2484-5a5b-40e9-8453-0e47fe060eff None None] Set router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:34:13.598872 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6e3e8c5-5008-45ca-a372-985cba4c6fc2 req-79b36e80-e05b-479a-abfe-cd46d4d426b4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:34:13.600598 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b6e3e8c5-5008-45ca-a372-985cba4c6fc2 req-79b36e80-e05b-479a-abfe-cd46d4d426b4 service neutron] 10.222.0.93 "GET /v2.0/ports?tenant_id=4ee563f50afe4088aebf60ed28c59ea7&device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 1099 time: 0.0949962 May 24 11:34:13.759092 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6e3e8c5-5008-45ca-a372-985cba4c6fc2 req-30342ccf-d379-4488-b062-f27426c0f42f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:34:13.760555 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b6e3e8c5-5008-45ca-a372-985cba4c6fc2 req-30342ccf-d379-4488-b062-f27426c0f42f service neutron] 10.222.0.93 "GET /v2.0/networks?id=496a7264-ce0a-4dd1-b29d-67154eb68a39 HTTP/1.1" status: 200 len: 909 time: 0.1515758 May 24 11:34:13.773519 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f7502118-be5e-4920-aac3-4be4cf774792 req-a4c68a98-c4c6-4094-b6d1-47f63ff65584 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91820) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:34:13.775036 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f7502118-be5e-4920-aac3-4be4cf774792 req-a4c68a98-c4c6-4094-b6d1-47f63ff65584 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.0926349 May 24 11:34:13.826223 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [req-f7502118-be5e-4920-aac3-4be4cf774792 req-d8ef5191-69e7-4797-959e-3d33d69b70bf tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0450301 May 24 11:34:13.837917 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b6e3e8c5-5008-45ca-a372-985cba4c6fc2 req-a9250b99-7fcd-4f5d-88f1-42014841d072 service neutron] 10.222.0.93 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=66674db0-9a62-4491-be8b-38bf92ccfa58 HTTP/1.1" status: 200 len: 217 time: 0.0687492 May 24 11:34:13.884468 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-6a9e2484-5a5b-40e9-8453-0e47fe060eff None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 4, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:51Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'a61e4d23-b4eb-4110-a731-0d50771b3303', 'name': '', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'mac_address': 'fa:16:3e:f6:4c:6d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_interface', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '86fc06b2-ed33-40b0-849c-081575fab447', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:47Z', 'updated_at': '2021-05-24T11:29:55Z', 'revision_number': 5, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'subnets': [{'id': '86fc06b2-ed33-40b0-849c-081575fab447', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:13.888418 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6a9e2484-5a5b-40e9-8453-0e47fe060eff None None] Checking router: 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:34:13.897114 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6a9e2484-5a5b-40e9-8453-0e47fe060eff None None] Routers {'2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:34:13.897749 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-6a9e2484-5a5b-40e9-8453-0e47fe060eff None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:33:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:13.906894 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6e3e8c5-5008-45ca-a372-985cba4c6fc2 req-f96ec9fc-59e5-47c4-917d-af87441e306d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:34:13.908356 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b6e3e8c5-5008-45ca-a372-985cba4c6fc2 req-f96ec9fc-59e5-47c4-917d-af87441e306d service neutron] 10.222.0.93 "GET /v2.0/subnets?id=86fc06b2-ed33-40b0-849c-081575fab447 HTTP/1.1" status: 200 len: 847 time: 0.0620980 May 24 11:34:13.987685 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6e3e8c5-5008-45ca-a372-985cba4c6fc2 req-27a0e7ce-e14e-4967-a6af-4feb17d13d1b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:34:13.989231 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b6e3e8c5-5008-45ca-a372-985cba4c6fc2 req-27a0e7ce-e14e-4967-a6af-4feb17d13d1b service neutron] 10.222.0.93 "GET /v2.0/ports?network_id=496a7264-ce0a-4dd1-b29d-67154eb68a39&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0736384 May 24 11:34:14.085940 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-6a9e2484-5a5b-40e9-8453-0e47fe060eff None None] Set router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:34:14.134155 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b6e3e8c5-5008-45ca-a372-985cba4c6fc2 req-962b23df-5aea-47e6-8439-67f15ddfed90 service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1368060 May 24 11:34:14.270768 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-b6e3e8c5-5008-45ca-a372-985cba4c6fc2 req-10697d6b-a6b4-46eb-ab98-b54998e7a20a service neutron] 10.222.0.93 "GET /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1294947 May 24 11:34:14.399394 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-6a9e2484-5a5b-40e9-8453-0e47fe060eff None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 4, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:51Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'a61e4d23-b4eb-4110-a731-0d50771b3303', 'name': '', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'mac_address': 'fa:16:3e:f6:4c:6d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_interface', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '86fc06b2-ed33-40b0-849c-081575fab447', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:47Z', 'updated_at': '2021-05-24T11:29:55Z', 'revision_number': 5, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'subnets': [{'id': '86fc06b2-ed33-40b0-849c-081575fab447', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:14.403632 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6a9e2484-5a5b-40e9-8453-0e47fe060eff None None] The sync data for ids ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'] in dv-cinder-784092-9-iscsi is: [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:29:48Z', 'revision_number': 4, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:51Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi', '_interfaces': [{'id': 'a61e4d23-b4eb-4110-a731-0d50771b3303', 'name': '', 'network_id': '496a7264-ce0a-4dd1-b29d-67154eb68a39', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'mac_address': 'fa:16:3e:f6:4c:6d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_interface', 'standard_attr_id': 663, 'fixed_ips': [{'subnet_id': '86fc06b2-ed33-40b0-849c-081575fab447', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:47Z', 'updated_at': '2021-05-24T11:29:55Z', 'revision_number': 5, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'subnets': [{'id': '86fc06b2-ed33-40b0-849c-081575fab447', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:34:14.608485 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-edbc3656-127d-4251-afc7-2f55fdcd0608 None None] New status for floating IP c6b3f275-b04b-4c9b-8d54-42d4849eb32a: DOWN {{(pid=91822) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 24 11:34:15.191733 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5370e348-d13d-45a2-ae71-c1663db99f99 req-91decb93-74a7-4837-a43f-e0109d15fa0e tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:34:15.192422 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-5370e348-d13d-45a2-ae71-c1663db99f99 req-91decb93-74a7-4837-a43f-e0109d15fa0e tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Synchronizing usage tracker for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:34:15.210195 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-78f5a2ee-d8dc-4b77-8f23-b6525ff962b1 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91819) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 24 11:34:15.211412 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-78f5a2ee-d8dc-4b77-8f23-b6525ff962b1 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Synchronizing usage tracker for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:34:15.226702 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-5370e348-d13d-45a2-ae71-c1663db99f99 req-91decb93-74a7-4837-a43f-e0109d15fa0e tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Unset dirty status for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:34:15.227861 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5370e348-d13d-45a2-ae71-c1663db99f99 req-91decb93-74a7-4837-a43f-e0109d15fa0e tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.2926481 May 24 11:34:15.230320 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-78f5a2ee-d8dc-4b77-8f23-b6525ff962b1 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Unset dirty status for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:34:15.231312 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-78f5a2ee-d8dc-4b77-8f23-b6525ff962b1 tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.93 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 946 time: 0.1364684 May 24 11:34:15.325632 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-5370e348-d13d-45a2-ae71-c1663db99f99 req-99a688ed-3e30-4d94-9110-9e5ee11894dc tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/security-groups?id=225a506e-ecb3-47d8-9bd2-46faece5caa7&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0914710 May 24 11:34:15.416370 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-ef7287d7-c79f-4192-857b-1f9230e3dfae tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Deleting port 66674db0-9a62-4491-be8b-38bf92ccfa58 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:34:15.416735 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-ef7287d7-c79f-4192-857b-1f9230e3dfae tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:15.561671 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-ef7287d7-c79f-4192-857b-1f9230e3dfae tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] For port 66674db0-9a62-4491-be8b-38bf92ccfa58, host hv-cinder-78409, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78409',level=0,port_id=66674db0-9a62-4491-be8b-38bf92ccfa58,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:34:15.562036 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-ef7287d7-c79f-4192-857b-1f9230e3dfae tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:15.691263 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-ef7287d7-c79f-4192-857b-1f9230e3dfae tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Calling delete_port for 66674db0-9a62-4491-be8b-38bf92ccfa58 owned by compute:nova {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:34:15.806800 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-ef7287d7-c79f-4192-857b-1f9230e3dfae tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:15.813015 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-d2939fd7-4b43-4396-ab59-1f6b850df9e8 tempest-VolumesNegativeTest-903119013 tempest-VolumesNegativeTest-903119013-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:15.863216 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ef7287d7-c79f-4192-857b-1f9230e3dfae tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event deleted for resources: {'Port': ['ID=66674db0-9a62-4491-be8b-38bf92ccfa58,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:15.866253 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-ef7287d7-c79f-4192-857b-1f9230e3dfae tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:15.946431 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-ef7287d7-c79f-4192-857b-1f9230e3dfae tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] No DVR routers for this DVR port 66674db0-9a62-4491-be8b-38bf92ccfa58 on host hv-cinder-78409 {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:34:15.949047 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-ef7287d7-c79f-4192-857b-1f9230e3dfae tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-126854'] for port, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:15.951707 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'name': 'network-vif-deleted', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58'}] {{(pid=91819) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 24 11:34:15.968298 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-6ced2900-f5f3-4bfc-82f4-1ead775cbb5b req-ef7287d7-c79f-4192-857b-1f9230e3dfae tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.93 "DELETE /v2.0/ports/66674db0-9a62-4491-be8b-38bf92ccfa58 HTTP/1.1" status: 204 len: 173 time: 0.7211106 May 24 11:34:15.998896 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7', 'name': 'network-vif-deleted', 'tag': '66674db0-9a62-4491-be8b-38bf92ccfa58', 'status': 'completed', 'code': 200} May 24 11:34:16.484925 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-d2a6e52d-9c32-4863-a8ac-4fcd07f10878 req-ebad4b4f-6514-475d-9fd4-0ce4b515620d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Synchronizing usage tracker for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:port {{(pid=91819) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 24 11:34:16.503093 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.quota.resource [req-d2a6e52d-9c32-4863-a8ac-4fcd07f10878 req-ebad4b4f-6514-475d-9fd4-0ce4b515620d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Unset dirty status for tenant:4ee563f50afe4088aebf60ed28c59ea7 on resource:port {{(pid=91819) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 24 11:34:16.506928 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [req-d2a6e52d-9c32-4863-a8ac-4fcd07f10878 req-ebad4b4f-6514-475d-9fd4-0ce4b515620d tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "GET /v2.0/ports?device_id=a6a6d239-c9cc-44f7-a0ac-7f16f1ca10a7 HTTP/1.1" status: 200 len: 210 time: 0.0690408 May 24 11:34:17.603892 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 35014) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:17.677397 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:17.707959 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:17.779323 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:17.783097 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-84090ff6-64f3-4a36-8d2b-ad07c435a21b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:17.798786 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=225a506e-ecb3-47d8-9bd2-46faece5caa7,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:17.807793 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:17.808165 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:17.808531 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:17.808889 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:17.809279 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:17.809628 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:17.809976 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:17.810322 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:17.810667 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:17.811009 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:17.811381 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:17.811719 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91820) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:17.817323 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:17.821314 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:17.822442 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-014aaaad-ec0f-42ed-815f-5e2ac1caf55b tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] 10.222.0.18 "DELETE /v2.0/security-groups/225a506e-ecb3-47d8-9bd2-46faece5caa7 HTTP/1.1" status: 204 len: 168 time: 0.2171865 May 24 11:34:17.867701 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 35020) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:17.946011 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 66674db0-9a62-4491-be8b-38bf92ccfa58 no longer exists at agent hyperv_hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:34:17.985184 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] No Port match for: 66674db0-9a62-4491-be8b-38bf92ccfa58 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:34:17.985612 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Device 66674db0-9a62-4491-be8b-38bf92ccfa58 not bound to the agent host hv-cinder-78409 {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:34:18.418550 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-687465'] for router_interface, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:18.949313 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] Deleting port a61e4d23-b4eb-4110-a731-0d50771b3303 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:34:18.950048 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:19.075399 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] For port a61e4d23-b4eb-4110-a731-0d50771b3303, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=a61e4d23-b4eb-4110-a731-0d50771b3303,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:34:19.075752 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:19.204941 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] Calling delete_port for a61e4d23-b4eb-4110-a731-0d50771b3303 owned by network:router_interface {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:34:19.348079 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:19.352686 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c4205ad0-773b-46b6-878c-a87283a497ea admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:19.399829 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] Pushing event deleted for resources: {'Port': ['ID=a61e4d23-b4eb-4110-a731-0d50771b3303,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:19.402458 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:19.686515 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-537767', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-685561'] for router_interface, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:20.174722 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] Router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:34:20.360194 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:34:20.368587 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2d4b80fd-5b41-4f9f-b1e0-80cbf18fb7ec None None] Sync routers for ids ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'] in dv-cinder-784092-9-iscsi {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:34:20.377355 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] 10.222.0.18 "PUT /v2.0/routers/2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.5084147 May 24 11:34:20.378239 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2d4b80fd-5b41-4f9f-b1e0-80cbf18fb7ec None None] Routers {'2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:34:20.378862 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-2d4b80fd-5b41-4f9f-b1e0-80cbf18fb7ec None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:33:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:20.379805 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 35024) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:20.557712 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Notify callbacks [] for router, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:20.562617 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-2d4b80fd-5b41-4f9f-b1e0-80cbf18fb7ec None None] Set router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:34:20.735201 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:20.811895 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Deleting port 0b304a09-229c-496f-b22c-42da5ad3c052 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:34:20.812102 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:20.853283 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-2d4b80fd-5b41-4f9f-b1e0-80cbf18fb7ec None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185'}], 'enable_snat': True}, 'gw_port_id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:34:19Z', 'revision_number': 5, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port': {'id': '0b304a09-229c-496f-b22c-42da5ad3c052', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:7c:f3:3b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 662, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.185', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:45Z', 'updated_at': '2021-05-24T11:29:51Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:20.857110 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2d4b80fd-5b41-4f9f-b1e0-80cbf18fb7ec None None] Checking router: 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 for host: dv-cinder-784092-9-iscsi {{(pid=91821) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:34:20.866066 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2d4b80fd-5b41-4f9f-b1e0-80cbf18fb7ec None None] Routers {'2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91821) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:34:20.866711 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-2d4b80fd-5b41-4f9f-b1e0-80cbf18fb7ec None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:33:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'], 'active': True} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:20.945336 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] For port 0b304a09-229c-496f-b22c-42da5ad3c052, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=0b304a09-229c-496f-b22c-42da5ad3c052,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:34:20.945693 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:21.063873 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvr_db [None req-2d4b80fd-5b41-4f9f-b1e0-80cbf18fb7ec None None] Set router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 gateway port host: None {{(pid=91821) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:34:21.076585 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Calling delete_port for 0b304a09-229c-496f-b22c-42da5ad3c052 owned by network:router_gateway {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:34:21.178354 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_hamode_db [None req-2d4b80fd-5b41-4f9f-b1e0-80cbf18fb7ec None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:34:20Z', 'revision_number': 6, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port_host': None}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91821) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:21.182726 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2d4b80fd-5b41-4f9f-b1e0-80cbf18fb7ec None None] The sync data for ids ['2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0'] in dv-cinder-784092-9-iscsi is: [{'id': '2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0', 'name': 'tempest-TestShelveInstance-478081922-router', 'tenant_id': '4ee563f50afe4088aebf60ed28c59ea7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:44Z', 'updated_at': '2021-05-24T11:34:20Z', 'revision_number': 6, 'project_id': '4ee563f50afe4088aebf60ed28c59ea7', 'gw_port_host': None}] {{(pid=91821) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:34:21.226749 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:21.230840 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:21.275400 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Pushing event deleted for resources: {'Port': ['ID=0b304a09-229c-496f-b22c-42da5ad3c052,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:21.278274 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:21.369597 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-686964'] for router_gateway, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:21.668047 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-684537', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4171'] for router, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:21.670245 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:21.693210 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-684229'] for router, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:21.693824 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 {{(pid=91819) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 11:34:21.705735 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] Notify callbacks [] for router, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:21.706851 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-032d6fbf-5c68-4d04-a710-0e09451c4fa1 admin admin] 10.222.0.18 "DELETE /v2.0/routers/2aa3fc07-9f53-4bb9-a5a2-34c653ae43b0 HTTP/1.1" status: 204 len: 168 time: 1.3261158 May 24 11:34:21.709446 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 35028) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:21.768151 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_v2 [None req-e8a2a4fa-1bfe-4d56-b61f-3e44f1328fb5 admin admin] Deleting subnet 86fc06b2-ed33-40b0-849c-081575fab447 {{(pid=91819) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:34:21.771361 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e8a2a4fa-1bfe-4d56-b61f-3e44f1328fb5 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:21.824939 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e8a2a4fa-1bfe-4d56-b61f-3e44f1328fb5 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534853172'] for subnet, precommit_delete_associations {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:21.902074 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e8a2a4fa-1bfe-4d56-b61f-3e44f1328fb5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.164106 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.172796 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e8a2a4fa-1bfe-4d56-b61f-3e44f1328fb5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534944900', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535570805'] for subnet, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.176891 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:22.209998 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e8a2a4fa-1bfe-4d56-b61f-3e44f1328fb5 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=86fc06b2-ed33-40b0-849c-081575fab447,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:22.213945 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e8a2a4fa-1bfe-4d56-b61f-3e44f1328fb5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:22.216775 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e8a2a4fa-1bfe-4d56-b61f-3e44f1328fb5 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.221825 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d744d473-78a1-44f4-b67b-4380fc7cd76d None None] Network 496a7264-ce0a-4dd1-b29d-67154eb68a39 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:34:22.225046 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e8a2a4fa-1bfe-4d56-b61f-3e44f1328fb5 admin admin] 10.222.0.18 "DELETE /v2.0/subnets/86fc06b2-ed33-40b0-849c-081575fab447 HTTP/1.1" status: 204 len: 168 time: 0.5122781 May 24 11:34:22.225233 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 35032) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:22.331173 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.408812 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Deleting port 0c7b8795-f4a5-424e-8b50-5a5d254488ad {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:34:22.409181 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.485229 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] DHCP port deletion for 496a7264-ce0a-4dd1-b29d-67154eb68a39 request from dv-cinder-784092-9-iscsi {{(pid=91821) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 11:34:22.510565 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] For port 0c7b8795-f4a5-424e-8b50-5a5d254488ad, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=0c7b8795-f4a5-424e-8b50-5a5d254488ad,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:34:22.510924 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.550581 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Deleting port 0c7b8795-f4a5-424e-8b50-5a5d254488ad {{(pid=91821) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:34:22.550963 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.639521 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Calling delete_port for 0c7b8795-f4a5-424e-8b50-5a5d254488ad owned by network:dhcp {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:34:22.652692 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] For port 0c7b8795-f4a5-424e-8b50-5a5d254488ad, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=0c7b8795-f4a5-424e-8b50-5a5d254488ad,segment=NetworkSegment(417bf2a8-a5cf-497b-a631-e0f581048962),segment_id=417bf2a8-a5cf-497b-a631-e0f581048962)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:34:22.653185 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.688163 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.692142 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-baeb3428-ff84-4910-95d7-dc8f65e3f3be admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:22.734580 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Pushing event deleted for resources: {'Port': ['ID=0c7b8795-f4a5-424e-8b50-5a5d254488ad,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:22.738201 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:22.740412 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] No DVR routers for this DVR port 0c7b8795-f4a5-424e-8b50-5a5d254488ad on host dv-cinder-784092-9-iscsi {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:34:22.775400 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Calling delete_port for 0c7b8795-f4a5-424e-8b50-5a5d254488ad owned by network:dhcp {{(pid=91821) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:34:22.818072 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093'] for port, after_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.822402 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:22.858058 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.871492 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['496a7264-ce0a-4dd1-b29d-67154eb68a39']}} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:22.872888 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] No DVR routers for this DVR port 0c7b8795-f4a5-424e-8b50-5a5d254488ad on host dv-cinder-784092-9-iscsi {{(pid=91821) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:34:22.882966 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Pushing event deleted for resources: {'Port': ['ID=0c7b8795-f4a5-424e-8b50-5a5d254488ad,revision_number=None']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:22.886163 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:22.886502 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:22.886834 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:22.887159 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:22.887479 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:22.887797 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:22.888112 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:22.888432 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:22.888748 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:22.889053 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:22.889387 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:22.889621 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '417bf2a8-a5cf-497b-a631-e0f581048962') {'for_net_delete': True} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:22.889745 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-784092-9-iscsi {{(pid=91821) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 24 11:34:22.890141 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '417bf2a8-a5cf-497b-a631-e0f581048962') {} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:22.892796 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-aa78f641-9faa-4631-a024-0cb7ba1319da None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:22.911505 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534853183'] for segment, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.943812 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.954846 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Releasing vlan 540 on physical network physnet1 to pool {{(pid=91819) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:34:22.955508 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535155490'] for segment, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:22.962774 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device a61e4d23-b4eb-4110-a731-0d50771b3303 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: Traceback (most recent call last): May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: timer() May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: cb(*args, **kw) May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: return func(*args, **kwargs) May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: self._notify() May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: self.callback(batched_events) May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: event.method(event) May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: return f(self, *a, **k) May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._get(url).json() May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: return self.request(url, 'GET', **kwargs) May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:34:22.985654 dv-cinder-784092-9-iscsi neutron-server[91819]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:34:23.019115 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: a61e4d23-b4eb-4110-a731-0d50771b3303 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:34:23.019477 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device a61e4d23-b4eb-4110-a731-0d50771b3303 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:34:23.019936 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 0c7b8795-f4a5-424e-8b50-5a5d254488ad no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:34:23.053274 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 0c7b8795-f4a5-424e-8b50-5a5d254488ad {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:34:23.053623 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 0c7b8795-f4a5-424e-8b50-5a5d254488ad not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:34:23.054554 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 0b304a09-229c-496f-b22c-42da5ad3c052 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:34:23.061514 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:23.065322 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:23.079030 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:23.089638 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Pushing event deleted for resources: {'Network': ['ID=496a7264-ce0a-4dd1-b29d-67154eb68a39,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:23.092795 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:23.095030 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 0b304a09-229c-496f-b22c-42da5ad3c052 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:34:23.095346 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 0b304a09-229c-496f-b22c-42da5ad3c052 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:34:23.105074 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-75ce8754-6f89-467d-bc65-557b14a0d30a admin admin] 10.222.0.18 "DELETE /v2.0/networks/496a7264-ce0a-4dd1-b29d-67154eb68a39 HTTP/1.1" status: 204 len: 168 time: 0.8783000 May 24 11:34:23.107927 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 35036) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:23.474029 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-11da8396-d1a8-4e96-9255-984e7e952517 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:23.788041 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-687465'] for router_interface, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:24.203541 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] Deleting port 4501d99c-a6db-411c-8f0c-fb483478d502 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:34:24.203887 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:24.325588 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] For port 4501d99c-a6db-411c-8f0c-fb483478d502, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=4501d99c-a6db-411c-8f0c-fb483478d502,segment=NetworkSegment(f444ecab-e99d-43ff-8216-802ff480029d),segment_id=f444ecab-e99d-43ff-8216-802ff480029d)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:34:24.325942 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:24.449082 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] Calling delete_port for 4501d99c-a6db-411c-8f0c-fb483478d502 owned by network:router_interface {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:34:24.588693 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:24.592160 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-9f220f65-21f1-4611-8ef3-c5c71d9f81da admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:24.636771 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] Pushing event deleted for resources: {'Port': ['ID=4501d99c-a6db-411c-8f0c-fb483478d502,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:24.663742 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:24.900440 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-537767', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-685561'] for router_interface, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:25.377014 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] Router 2ab2d866-7255-4b75-be59-27b23905b387 has already been hosted by L3 agent f02aeb5a-cb08-4702-b9ef-bf6da41af174 {{(pid=91819) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 24 11:34:25.563192 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] Notify agent at l3_agent.dv-cinder-784092-9-iscsi the message routers_updated {{(pid=91819) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 24 11:34:25.571705 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-805ac070-9ec6-49e4-8d15-3b3b04943625 None None] Sync routers for ids ['2ab2d866-7255-4b75-be59-27b23905b387'] in dv-cinder-784092-9-iscsi {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 24 11:34:25.580106 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-529ef60b-399a-4f52-853b-4fb86974a189 admin admin] 10.222.0.18 "PUT /v2.0/routers/2ab2d866-7255-4b75-be59-27b23905b387/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.4711709 May 24 11:34:25.581898 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-805ac070-9ec6-49e4-8d15-3b3b04943625 None None] Routers {'2ab2d866-7255-4b75-be59-27b23905b387'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:34:25.582504 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-805ac070-9ec6-49e4-8d15-3b3b04943625 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:33:56Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2ab2d866-7255-4b75-be59-27b23905b387'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:25.582728 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 35040) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:25.763745 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-805ac070-9ec6-49e4-8d15-3b3b04943625 None None] Set router 2ab2d866-7255-4b75-be59-27b23905b387 gateway port host: dv-cinder-784092-9-iscsi {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:34:25.764094 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Notify callbacks [] for router, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:25.806140 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:25.905617 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-cc9bc3d8-943b-40d2-9632-8f04260d87a7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:25.942121 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:26.016809 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Deleting port aece7b82-6ccb-4ae5-be76-eaf6f07278b1 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:34:26.017053 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:26.060386 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-805ac070-9ec6-49e4-8d15-3b3b04943625 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2ab2d866-7255-4b75-be59-27b23905b387', 'name': 'tempest-TestShelveInstance-1223486564-router', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'external_fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193'}], 'enable_snat': True}, 'gw_port_id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:51Z', 'updated_at': '2021-05-24T11:34:24Z', 'revision_number': 5, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'gw_port': {'id': 'aece7b82-6ccb-4ae5-be76-eaf6f07278b1', 'name': '', 'network_id': '858b8fad-a73f-4578-9079-6e731b9ff7e2', 'tenant_id': '', 'mac_address': 'fa:16:3e:28:2e:ea', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2ab2d866-7255-4b75-be59-27b23905b387', 'device_owner': 'network:router_gateway', 'standard_attr_id': 674, 'fixed_ips': [{'subnet_id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'ip_address': '172.27.0.193', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-784092-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-24T11:29:52Z', 'updated_at': '2021-05-24T11:29:59Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'a677ede7-728d-486b-819a-5fc603fb3f9b', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-784092-9-iscsi'}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:26.064662 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-805ac070-9ec6-49e4-8d15-3b3b04943625 None None] Checking router: 2ab2d866-7255-4b75-be59-27b23905b387 for host: dv-cinder-784092-9-iscsi {{(pid=91822) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 24 11:34:26.072928 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvrscheduler_db [None req-805ac070-9ec6-49e4-8d15-3b3b04943625 None None] Routers {'2ab2d866-7255-4b75-be59-27b23905b387'} bound to L3 agent in host dv-cinder-784092-9-iscsi {{(pid=91822) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 24 11:34:26.073570 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-805ac070-9ec6-49e4-8d15-3b3b04943625 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-784092-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 2, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-24T11:07:56Z,description=,heartbeat_timestamp=2021-05-24T11:34:26Z,host='dv-cinder-784092-9-iscsi',id=f02aeb5a-cb08-4702-b9ef-bf6da41af174,load=0,resource_versions=,resources_synced=,started_at=2021-05-24T11:07:56Z,topic='l3_agent')) {'router_ids': ['2ab2d866-7255-4b75-be59-27b23905b387'], 'active': True} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:26.141769 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] For port aece7b82-6ccb-4ae5-be76-eaf6f07278b1, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=aece7b82-6ccb-4ae5-be76-eaf6f07278b1,segment=NetworkSegment(eef8fa6c-c77b-4b44-a377-fa3b8b4035b0),segment_id=eef8fa6c-c77b-4b44-a377-fa3b8b4035b0)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:34:26.142120 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:26.254485 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_dvr_db [None req-805ac070-9ec6-49e4-8d15-3b3b04943625 None None] Set router 2ab2d866-7255-4b75-be59-27b23905b387 gateway port host: None {{(pid=91822) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 24 11:34:26.263123 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Calling delete_port for aece7b82-6ccb-4ae5-be76-eaf6f07278b1 owned by network:router_gateway {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:34:26.358329 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.l3_hamode_db [None req-805ac070-9ec6-49e4-8d15-3b3b04943625 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2ab2d866-7255-4b75-be59-27b23905b387', 'name': 'tempest-TestShelveInstance-1223486564-router', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:51Z', 'updated_at': '2021-05-24T11:34:25Z', 'revision_number': 6, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'gw_port_host': None}], 'dv-cinder-784092-9-iscsi', False) {} {{(pid=91822) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:26.362383 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-805ac070-9ec6-49e4-8d15-3b3b04943625 None None] The sync data for ids ['2ab2d866-7255-4b75-be59-27b23905b387'] in dv-cinder-784092-9-iscsi is: [{'id': '2ab2d866-7255-4b75-be59-27b23905b387', 'name': 'tempest-TestShelveInstance-1223486564-router', 'tenant_id': '5d1d48e7fd374a08ae78e1b121cea102', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-24T11:29:51Z', 'updated_at': '2021-05-24T11:34:25Z', 'revision_number': 6, 'project_id': '5d1d48e7fd374a08ae78e1b121cea102', 'gw_port_host': None}] {{(pid=91822) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 24 11:34:26.402519 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:26.406934 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-5ee3d08f-c553-4f8a-a093-85779ba092bf service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:26.475249 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Pushing event deleted for resources: {'Port': ['ID=aece7b82-6ccb-4ae5-be76-eaf6f07278b1,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:26.481748 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:26.664370 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-686964'] for router_gateway, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:26.838958 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-684537', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-4171'] for router, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:26.841077 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:26.864559 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-684229'] for router, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:26.865061 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Fanout notify agent at l3_agent the message router_deleted on router 2ab2d866-7255-4b75-be59-27b23905b387 {{(pid=91819) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 24 11:34:26.879858 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] Notify callbacks [] for router, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:26.880979 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e7c6a897-efc0-411b-b525-1c26074a607f admin admin] 10.222.0.18 "DELETE /v2.0/routers/2ab2d866-7255-4b75-be59-27b23905b387 HTTP/1.1" status: 204 len: 168 time: 1.2974834 May 24 11:34:26.883537 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 35044) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:26.948718 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.db_base_plugin_v2 [None req-e0e34862-94dd-4ca7-b323-3261e75ccfa6 admin admin] Deleting subnet 4a49f00e-a88d-4923-829e-bbe760c82a9a {{(pid=91819) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 24 11:34:26.952708 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e0e34862-94dd-4ca7-b323-3261e75ccfa6 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:27.005122 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e0e34862-94dd-4ca7-b323-3261e75ccfa6 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-534853172'] for subnet, precommit_delete_associations {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:27.051456 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:27.096934 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e0e34862-94dd-4ca7-b323-3261e75ccfa6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-1893991'] for subnet, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:27.146394 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 4501d99c-a6db-411c-8f0c-fb483478d502 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:34:27.181942 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: 4501d99c-a6db-411c-8f0c-fb483478d502 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:34:27.182564 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device 4501d99c-a6db-411c-8f0c-fb483478d502 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:34:27.183171 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device aece7b82-6ccb-4ae5-be76-eaf6f07278b1 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:34:27.220311 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: aece7b82-6ccb-4ae5-be76-eaf6f07278b1 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:34:27.220722 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device aece7b82-6ccb-4ae5-be76-eaf6f07278b1 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:34:27.371471 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e0e34862-94dd-4ca7-b323-3261e75ccfa6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-1892818', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511157', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-534944900', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-535570805'] for subnet, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:27.376692 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c5f08cf3-1e7a-4f85-a701-5fd3903fbf5f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:27.411019 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e0e34862-94dd-4ca7-b323-3261e75ccfa6 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=4a49f00e-a88d-4923-829e-bbe760c82a9a,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:27.414658 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-e0e34862-94dd-4ca7-b323-3261e75ccfa6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:27.415637 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-e0e34862-94dd-4ca7-b323-3261e75ccfa6 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for subnet, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:27.420409 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-1314004b-de92-4864-ab06-bc27f0502fb4 None None] Network e519e229-2b8f-4026-b816-b3ebd84b0979 requested from dv-cinder-784092-9-iscsi {{(pid=91821) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 24 11:34:27.421620 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-e0e34862-94dd-4ca7-b323-3261e75ccfa6 admin admin] 10.222.0.18 "DELETE /v2.0/subnets/4a49f00e-a88d-4923-829e-bbe760c82a9a HTTP/1.1" status: 204 len: 168 time: 0.5371544 May 24 11:34:27.424029 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 35048) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:27.530758 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-1741800'] for network, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:27.616526 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Deleting port f4050d5e-51aa-4176-ac19-2a9929cf87a6 {{(pid=91819) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:34:27.616881 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:27.666164 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-2d006a3a-8482-4ad6-b8ce-5ad84d58a0d5 None None] DHCP port deletion for e519e229-2b8f-4026-b816-b3ebd84b0979 request from dv-cinder-784092-9-iscsi {{(pid=91821) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 24 11:34:27.729327 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.db [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] For port f4050d5e-51aa-4176-ac19-2a9929cf87a6, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=f4050d5e-51aa-4176-ac19-2a9929cf87a6,segment=NetworkSegment(f444ecab-e99d-43ff-8216-802ff480029d),segment_id=f444ecab-e99d-43ff-8216-802ff480029d)] {{(pid=91819) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:34:27.729690 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:27.741395 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-2d006a3a-8482-4ad6-b8ce-5ad84d58a0d5 None None] Deleting port f4050d5e-51aa-4176-ac19-2a9929cf87a6 {{(pid=91821) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 24 11:34:27.741781 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-2d006a3a-8482-4ad6-b8ce-5ad84d58a0d5 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8779719269955'] for port, before_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:27.844871 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.db [None req-2d006a3a-8482-4ad6-b8ce-5ad84d58a0d5 None None] For port f4050d5e-51aa-4176-ac19-2a9929cf87a6, host dv-cinder-784092-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-784092-9-iscsi',level=0,port_id=f4050d5e-51aa-4176-ac19-2a9929cf87a6,segment=NetworkSegment(f444ecab-e99d-43ff-8216-802ff480029d),segment_id=f444ecab-e99d-43ff-8216-802ff480029d)] {{(pid=91821) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 24 11:34:27.845205 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-2d006a3a-8482-4ad6-b8ce-5ad84d58a0d5 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-688056'] for port, precommit_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:27.869209 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.plugin [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Calling delete_port for f4050d5e-51aa-4176-ac19-2a9929cf87a6 owned by network:dhcp {{(pid=91819) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:34:27.922963 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-535155575'] for port, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:27.926885 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-1e12229d-f43d-4393-8ea0-a7f7db7d2cfa tempest-TestShelveInstance-1937704891 tempest-TestShelveInstance-1937704891-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:27.971324 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Pushing event deleted for resources: {'Port': ['ID=f4050d5e-51aa-4176-ac19-2a9929cf87a6,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:27.974282 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:27.975066 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.db.l3_dvrscheduler_db [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] No DVR routers for this DVR port f4050d5e-51aa-4176-ac19-2a9929cf87a6 on host dv-cinder-784092-9-iscsi {{(pid=91819) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:34:28.015691 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-2d006a3a-8482-4ad6-b8ce-5ad84d58a0d5 None None] Calling delete_port for f4050d5e-51aa-4176-ac19-2a9929cf87a6 owned by network:dhcp {{(pid=91821) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 24 11:34:28.063747 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-2d006a3a-8482-4ad6-b8ce-5ad84d58a0d5 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-1952892', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-511111', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-537781', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8779719271846', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8779718821093'] for port, after_delete {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:28.067635 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-e05c812e-77e2-4350-9491-2099b044c968 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:28.094360 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-1892328', 'neutron.services.segments.db._delete_segments_for_network-8779719283685'] for network, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:28.107886 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['e519e229-2b8f-4026-b816-b3ebd84b0979']}} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:28.113360 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2d006a3a-8482-4ad6-b8ce-5ad84d58a0d5 None None] Pushing event deleted for resources: {'Port': ['ID=f4050d5e-51aa-4176-ac19-2a9929cf87a6,revision_number=None']} {{(pid=91821) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:28.115742 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG oslo_concurrency.lockutils [None req-2d006a3a-8482-4ad6-b8ce-5ad84d58a0d5 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91821) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:28.118743 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2d006a3a-8482-4ad6-b8ce-5ad84d58a0d5 None None] No DVR routers for this DVR port f4050d5e-51aa-4176-ac19-2a9929cf87a6 on host dv-cinder-784092-9-iscsi {{(pid=91821) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 24 11:34:28.122696 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'f444ecab-e99d-43ff-8216-802ff480029d') {'for_net_delete': True} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:28.122999 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.services.segments.db [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'f444ecab-e99d-43ff-8216-802ff480029d') {} {{(pid=91819) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 24 11:34:28.136085 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8779719292070', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-534853183'] for segment, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:28.157060 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272'] for segment, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:28.163781 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Releasing vlan 500 on physical network physnet1 to pool {{(pid=91819) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 24 11:34:28.164178 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-1896272', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-535155490'] for segment, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: Traceback (most recent call last): May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: timer() May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: cb(*args, **kw) May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: return func(*args, **kwargs) May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: self._notify() May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: self.callback(batched_events) May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: event.method(event) May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_id = self._get_aggregate_id(event.segment_id) May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: aggregate_uuid = self.p_client.list_aggregates( May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: return f(self, *a, **k) May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._get(url).json() May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: return self._client.get(url, endpoint_filter=self._ks_filter, May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: return self.request(url, 'GET', **kwargs) May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: base_url = self.get_endpoint(auth, allow=allow, May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: auth = self._auth_required(auth, 'determine endpoint URL') May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 24 11:34:28.283014 dv-cinder-784092-9-iscsi neutron-server[91819]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 24 11:34:28.359030 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-1892321', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-701414', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-537774', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-686953'] for network, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:28.362702 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-865b66d8-f342-48d7-9973-472d9641ad58 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:28.381625 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-537739'] for network, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:28.389822 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Pushing event deleted for resources: {'Network': ['ID=e519e229-2b8f-4026-b816-b3ebd84b0979,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:28.394187 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:28.398077 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-61baf0ec-0612-46b5-841e-a1600cd1ac16 admin admin] 10.222.0.18 "DELETE /v2.0/networks/e519e229-2b8f-4026-b816-b3ebd84b0979 HTTP/1.1" status: 204 len: 168 time: 0.9730854 May 24 11:34:28.566947 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 35054) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:28.615264 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-50352c9b-64bc-4dbf-8f4a-01b6887cc41c admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=4ee563f50afe4088aebf60ed28c59ea7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0469882 May 24 11:34:28.618291 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [-] (91820) accepted ('10.222.0.18', 35058) {{(pid=91820) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:28.677344 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-27647347-1360-4064-85c6-ead0bfbdc5a0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:28.703615 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-27647347-1360-4064-85c6-ead0bfbdc5a0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:28.757708 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-27647347-1360-4064-85c6-ead0bfbdc5a0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:28.761440 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-4afe96ec-9249-4330-8588-0ba0bcab381d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:28.777129 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-27647347-1360-4064-85c6-ead0bfbdc5a0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c8a3db81-e029-4b49-99b9-010fa99fdcdd,revision_number=None']} {{(pid=91820) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:28.779444 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG oslo_concurrency.lockutils [None req-27647347-1360-4064-85c6-ead0bfbdc5a0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91820) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:28.789325 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron_lib.callbacks.manager [None req-27647347-1360-4064-85c6-ead0bfbdc5a0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91820) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:28.790644 dv-cinder-784092-9-iscsi neutron-server[91820]: INFO neutron.wsgi [None req-27647347-1360-4064-85c6-ead0bfbdc5a0 admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/c8a3db81-e029-4b49-99b9-010fa99fdcdd HTTP/1.1" status: 204 len: 168 time: 0.1712694 May 24 11:34:28.951087 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device f4050d5e-51aa-4176-ac19-2a9929cf87a6 no longer exists at agent ovs-agent-dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 24 11:34:28.993712 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.plugin [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] No Port match for: f4050d5e-51aa-4176-ac19-2a9929cf87a6 {{(pid=91821) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 24 11:34:28.994071 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron.plugins.ml2.rpc [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Device f4050d5e-51aa-4176-ac19-2a9929cf87a6 not bound to the agent host dv-cinder-784092-9-iscsi {{(pid=91821) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 24 11:34:29.007190 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:29.239207 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 35066) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:29.289152 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-68057233-0014-460b-864f-544307ac384c admin admin] 10.222.0.18 "GET /v2.0/security-groups?tenant_id=5d1d48e7fd374a08ae78e1b121cea102&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0486226 May 24 11:34:29.291487 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [-] (91819) accepted ('10.222.0.18', 35070) {{(pid=91819) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 24 11:34:29.349413 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c62505e5-617d-4ecf-8cda-878f8313f08f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:29.374285 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c62505e5-617d-4ecf-8cda-878f8313f08f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:29.431570 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c62505e5-617d-4ecf-8cda-878f8313f08f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-701410'] for security_group, after_delete {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:29.438151 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-745823f6-e7cb-4386-a1e5-dfe1eebacaff admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 24 11:34:29.451575 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c62505e5-617d-4ecf-8cda-878f8313f08f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=96201801-dd46-4721-be15-94e06c80399f,revision_number=None']} {{(pid=91819) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 24 11:34:29.454143 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG oslo_concurrency.lockutils [None req-c62505e5-617d-4ecf-8cda-878f8313f08f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91819) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 24 11:34:29.455329 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron_lib.callbacks.manager [None req-c62505e5-617d-4ecf-8cda-878f8313f08f admin admin] Notify callbacks [] for security_group, before_response {{(pid=91819) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:29.456419 dv-cinder-784092-9-iscsi neutron-server[91819]: INFO neutron.wsgi [None req-c62505e5-617d-4ecf-8cda-878f8313f08f admin admin] 10.222.0.18 "DELETE /v2.0/security-groups/96201801-dd46-4721-be15-94e06c80399f HTTP/1.1" status: 204 len: 168 time: 0.1640341 May 24 11:34:49.309015 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:34:52.170089 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:52.589248 dv-cinder-784092-9-iscsi neutron-server[91819]: DEBUG neutron.wsgi [req-fd828c2c-5157-485f-8240-52caed9508de req-9a56d6f7-3da5-4333-a574-4a38ab771841 service neutron] (91819) timed out ('10.222.0.93', 65223) {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 24 11:34:53.477593 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-786b0719-bd7f-4aa2-ae88-b3e2727601ba None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:55.814095 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:55.909880 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-10c80a23-8366-4411-90b1-4f7e9e917623 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:57.052090 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:34:58.997604 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:35:22.161066 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:35:23.480610 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-dbea971a-6747-48da-8f71-dd73fb1a520c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:35:25.808774 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:35:25.914986 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-7c44315c-2c96-4d87-972d-259ad37cb622 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:35:26.307670 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:35:27.056371 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:35:28.999272 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:35:52.159470 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:35:53.485247 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-e47caf28-9953-4bf6-a650-fd3aa148e796 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:35:55.821764 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:35:55.924240 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-956ab9e2-2d0b-41eb-98cc-6fbe49ffedbb None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:35:57.059000 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:35:59.003532 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:36:03.306408 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:36:22.169856 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:36:23.489199 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-93d61372-4d81-4bb8-a53b-09a69af74f54 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:36:25.817262 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:36:25.918772 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8798f6ac-7df1-4894-846a-9b08403608ca None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:36:27.063080 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:36:29.009833 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:36:40.305245 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:36:52.172581 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:36:53.492980 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-638cb553-4f66-486c-b4b5-d59130158435 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:36:55.821139 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:36:55.922130 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3fd13111-cd20-48e2-bbbc-774cdfc46d27 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:36:57.067520 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:36:59.007035 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:37:17.305771 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:37:22.177192 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:37:23.494877 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-292f84dc-57e1-4885-b835-adaf8203795d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:37:25.827264 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:37:25.924081 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-346c9981-02f0-4f78-8d5f-6c7e261bebf5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:37:27.067744 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:37:29.010438 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:37:52.195337 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:37:53.496573 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-5d5cf5f2-bcc1-4877-8230-22ec29b9ec45 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:37:54.304433 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:37:55.820577 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:37:55.927630 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a11fe4b7-946f-4b53-bcea-4465aecda624 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:37:57.069617 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:37:59.013678 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:38:09.250822 dv-cinder-784092-9-iscsi neutron-server[91820]: DEBUG neutron.wsgi [req-ce7fe2ef-0964-4f1d-b061-a6c98363ee5b req-9a2c13b1-56cc-468e-bad3-92000aa5e282 service neutron] (91820) timed out ('10.222.0.93', 65195) {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 24 11:38:22.201416 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:38:23.499007 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a308f790-6985-45f3-b099-daf271a3f9fb None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:38:25.822672 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:38:25.930343 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-8d1105d5-228d-4e1a-aba3-d09027d15816 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:38:27.074592 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:38:29.015277 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:38:31.303841 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:38:52.210658 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:38:53.501548 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aace8e0e-ce08-4311-b881-c42e5c368349 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:38:55.824436 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:38:55.931379 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-25fe75cb-454e-4b79-a899-d80b855b0f61 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:38:57.077298 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:38:59.019896 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:39:08.303837 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 24 11:39:22.230825 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-17e6922e-f185-4e15-ba43-322153cbd0ec None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:39:23.503691 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-7586ad9a-806a-4f55-879a-5b5fd4f83805 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:39:25.829071 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-aec1c29d-cde2-4492-aae3-22a9a6c98332 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:39:25.932647 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-a62373ab-61ff-4690-9199-6e4fdfb12f63 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:39:27.079929 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-3bf4d561-4722-40b8-b79d-6e748ad81595 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:39:29.021394 dv-cinder-784092-9-iscsi neutron-server[91821]: DEBUG neutron_lib.callbacks.manager [None req-64af8ae1-2307-471a-bb74-10b636e54d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8779719283703', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-1889830'] for agent, after_update {{(pid=91821) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 24 11:39:45.302808 dv-cinder-784092-9-iscsi neutron-server[91822]: DEBUG neutron.db.agents_db [None req-5c729ae6-a67d-4e08-96cd-5d6a1dc2bfb4 None None] Agent healthcheck: found 6 active agents {{(pid=91822) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}}